Device Driver Developer Interview Questions
Device Driver Developer Interview Questions and Answers
This page is for engineers preparing for device driver interviews: Linux kernel driver roles at silicon vendors, board and SoC teams, storage and networking companies, and the smaller number of Windows and RTOS driver jobs. It assumes you already write C comfortably and have at least built and loaded a kernel module. If you are still at that stage, work through the Device Driver Developer roadmap and the C Roadmap and Linux Roadmap roadmaps first.
Driver interviews usually have four parts. A C round tests pointer work, bit manipulation, data structures without a standard library, and concurrency. A kernel knowledge round walks through the driver model, interrupts, locking, DMA and power management, and the interviewer keeps asking "why" until you reach the edge of what you know. A debugging round hands you an oops, a lockdep splat or a symptom such as corrupted DMA data. A design round asks you to structure a driver for a real device. Senior loops add code review of a driver patch and questions about upstream work.
The page follows that order. Ten sections of questions run from the driver model through user-space interfaces, interrupts, locking, DMA, bus subsystems, power management, debugging and upstreaming, and finish with Windows and Zephyr. Then come eight C exercises that were compiled and tested, four open scenarios with model walk-throughs, a mock interview plan, a cheat sheet, red flags and an FAQ.
- Questions
- 60 with model answers
- Coding
- 8 exercises with solutions
- Scenarios
- 4 design and debugging rounds
How to use this page
- Read each question and answer it out loud before you open the model answer. Driver interviews are spoken, and the habit of naming the context (process, softirq or hard interrupt) first is what interviewers listen for.
- Do the coding exercises in a plain editor with
gcc -Wall -Wextra, then run them under-fsanitize=address,undefinedor-fsanitize=thread. Practise more timed problems on the coding practice page. - Before the interview, read a real driver in the kernel tree for a device class you claim to know, and be ready to explain its probe, interrupt and remove paths line by line.
- Practise reviewing code with the AI code review round round: driver code review questions reward the same habit of reading for contexts, lifetimes and error paths.
Kernel modules and the Linux driver model
Interviewers start here to check that you know how a driver binds to a device and who owns which object for how long.
What is the difference between a kernel module and a driver, and what happens when you load a module?Junior
A module is a unit of loadable kernel code; a driver is code that binds to devices. Most drivers are built as modules, but a driver can be built into the kernel, and many modules are not drivers at all (file systems, crypto algorithms, helpers).
When you run insmod or modprobe, the kernel copies the ELF object in, checks the version magic and, if enabled, the module signature, resolves its undefined symbols against exported kernel symbols, applies relocations and then calls the function registered with module_init(). For a driver that function usually just registers a struct pci_driver, platform_driver or similar. Registration makes the driver core walk existing devices on that bus and call probe() for each match, so probe can run before module_init() returns. modprobe differs from insmod in that it reads modules.dep and loads dependencies first.
Likely follow-up. What does module_platform_driver() expand to, and why is it preferred?
Explain major and minor numbers and the steps to register a character device.Junior
A dev_t holds a major and a minor number. Historically the major picked the driver and the minor picked the instance; today the kernel just maps a range of dev_t values to a struct cdev, and the driver uses the minor to find its per-device state.
The usual sequence is:
alloc_chrdev_region()to get a dynamic major and a block of minors.cdev_init()with yourfile_operations, thencdev_add()(orcdev_device_add(), which also adds thestruct device).class_create()once anddevice_create()per instance, so a uevent fires and udev or devtmpfs creates the/devnode.
Teardown runs in reverse. For a single simple node, a misc device (misc_register()) does all of this with one shared major. A strong answer also notes that cdev_add() makes the device live immediately, so all state the file operations need must be ready before that call.
Likely follow-up. Why should cdev_add() be one of the last steps in probe?
How does the bus, device and driver model decide to call probe()?Junior
Every device and every driver sits on a bus type (pci_bus_type, platform_bus_type, usb_bus_type and so on). When either a device or a driver is registered, the driver core calls the bus's match() callback for each candidate pair. The match uses whatever identity the bus has: vendor and device IDs for PCI, interface class and IDs for USB, and for platform devices the compatible string from device tree, an ACPI ID or a name.
On a match the core binds the pair and calls the bus probe(), which calls the driver's probe(). If probe returns 0 the device is bound, its driver pointer is set and it shows up under the driver in sysfs. If it returns -EPROBE_DEFER the core retries later. Any other error leaves the device unbound. You can bind and unbind by hand through the bind and unbind files in the driver's sysfs directory, which is also a quick way to test your remove path.
What do devm_ managed resources do, and when are they the wrong choice?Mid
devm_ functions record a release action on the device, and the driver core runs those actions in reverse order when probe fails or after remove() returns. They remove most error unwinding from probe and make leaks on failure paths rare.
They are wrong when the resource's lifetime is not the binding's lifetime:
- Memory reachable from an open file. If user space holds the file open after unbind, a
devm_kzalloc()buffer is freed whileread()can still touch it. Use a refcounted object freed inrelease(). - Ordering against non-managed teardown.
devm_request_irq()frees the interrupt afterremove()has run, so the handler can fire after you have stopped clocks or freed a workqueue thatremove()handled by hand. Either make everything managed in the right order or free the IRQ by hand. - Resources created outside probe, for example on every
open(), which would only be released at unbind.
The rule: devm_ is for things whose lifetime is exactly "while bound".
Likely follow-up. How would you add a custom cleanup step to the devm_ list?
How should probe() handle an error half-way through?Mid
Undo exactly what has been done, in reverse order, and return the error code, not a made-up one. Without devm_, the kernel idiom is a ladder of labels:
ret = clk_prepare_enable(priv->clk);
if (ret)
return ret;
ret = request_irq(irq, my_irq, 0, "mydev", priv);
if (ret)
goto err_clk;
ret = misc_register(&priv->misc);
if (ret)
goto err_irq;
return 0;
err_irq:
free_irq(irq, priv);
err_clk:
clk_disable_unprepare(priv->clk);
return ret;Two points interviewers look for. First, the user-visible registration (misc device, net device, input device) comes last, because once it exists user space can call into you. Second, use dev_err_probe() for errors that might be -EPROBE_DEFER: it stays quiet for deferral, records the reason for the deferred list and prints everything else.
How do you keep a character driver safe when the device is removed while a file is still open?Senior
Separate the lifetime of the hardware from the lifetime of your driver object. The device can go away at any time (unbind, hot unplug, module removal), but a file descriptor can stay open indefinitely, so the object that file->private_data points to must outlive the hardware.
The standard design is:
- Put a
struct kref(or embed thestruct deviceand use its refcount) in the per-device object.open()takes a reference,release()drops it, and the final put frees the object. - Keep a "gone" flag and the hardware pointers under a lock or
rw_semaphore.remove()takes the lock for write, sets the flag, clears the pointers and wakes any sleepers. Every file operation takes the lock for read and returns-ENODEVif the flag is set. - Use
cdev_device_add()andcdev_device_del()so the cdev cannot be opened after removal starts.
This is why devm_ memory must not hold state reachable from file operations.
Device tree, ACPI and platform resources
Board bring-up questions test whether you can describe hardware correctly and obtain clocks, regulators, pins and interrupts in the right order.
What is device tree, and how does a driver get matched to a node?Junior
Device tree is a data structure, passed by the boot loader, that describes hardware that cannot be discovered: which peripherals exist, their register addresses, interrupts, clocks, pins and how they connect. It is compiled from .dts and .dtsi sources into a binary blob (.dtb).
Matching uses the compatible property. A node lists one or more strings from most specific to most generic, such as "vendor,soc-uart", "vendor,generic-uart". The driver provides an of_device_id table with the strings it supports and points driver.of_match_table at it. The platform bus tries each compatible string in order, so a driver for the specific variant wins over a generic one. MODULE_DEVICE_TABLE(of, ...) exports the table so the module can be autoloaded. In probe, of_device_get_match_data() or device_get_match_data() returns the per-variant data attached to the matching entry, which is the clean way to handle small hardware differences.
How does a platform driver obtain its registers, interrupt, clock and regulator?Mid
Through the resource APIs, almost always the managed versions, in probe:
base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(base))
return PTR_ERR(base);
irq = platform_get_irq(pdev, 0);
if (irq < 0)
return irq;
clk = devm_clk_get(&pdev->dev, "core");
if (IS_ERR(clk))
return dev_err_probe(&pdev->dev, PTR_ERR(clk), "no clock\n");
vdd = devm_regulator_get(&pdev->dev, "vdd");Names such as "core" and "vdd" refer to clock-names and vdd-supply in the node. Simple properties come from device_property_read_u32() and friends, and GPIOs from devm_gpiod_get(). Two details matter: platform_get_irq() returns a negative error (never 0) on failure, so test < 0, and getting a clock does not enable it; you still call clk_prepare_enable() or use devm_clk_get_enabled().
Likely follow-up. What does devm_clk_get_optional() return when the clock is absent?
What makes a good device tree binding, and what do binding reviewers reject?Mid
A binding describes the hardware, not the driver's configuration or Linux internals. Reviewers reject properties such as "use DMA mode 2" or "buffer size", because those are software policy and would tie the binding to one operating system.
A good binding:
- Uses a specific compatible with a registered vendor prefix, plus a fallback only where the hardware really is compatible.
- Lists every resource the hardware has (registers, all interrupts, clocks, resets, supplies) even if today's driver does not use them all, because the binding is an ABI that outlives the driver.
- Uses standard property names and units (
-microvolt,-hz) and the common schemas for things like interrupts and pin control. - Is written as a YAML schema and passes
make dt_binding_check, with example DTS that passesdtbs_check.
Bindings go to the device tree maintainers as a separate patch, before the driver patch in the series.
How do you write one driver that works with both device tree and ACPI?Mid
Use the firmware-independent property API instead of of_ functions. device_property_read_u32(), device_property_present(), fwnode_ iterators and device_get_match_data() work against a fwnode_handle, which can be backed by a DT node, an ACPI node or software nodes. Resources such as memory, IRQs, GPIOs and clocks already come through generic platform and descriptor APIs.
Then add both match tables: an of_device_id table and an acpi_device_id table keyed by the ACPI _HID. ACPI can carry DT-style properties through _DSD, which lets one property name serve both. Where ACPI provides power control through methods such as _PS0 and _PS3, the ACPI power domain usually handles it, and the driver's runtime PM calls still work. Keep any #ifdef CONFIG_OF blocks out of the driver; the generic API makes them unnecessary.
Why should drivers use GPIO descriptors rather than GPIO numbers?Junior
The old integer API used a global GPIO number space that depended on probe order and board configuration, so numbers were fragile and drivers often hard-coded them. The descriptor API (gpiod_) gives the driver an opaque struct gpio_desc looked up by function name from DT or ACPI, such as reset-gpios.
The biggest practical gain is polarity. The DT flag GPIO_ACTIVE_LOW is part of the description, and gpiod_set_value() works in logical terms: setting 1 means "asserted", and the library inverts the line if the pin is active low. The driver writes gpiod_set_value(reset, 1) to assert reset whatever the board wiring. The raw functions exist when you really need the physical level. Use the _cansleep variants when the GPIO may sit behind an I2C expander, and request the line with the right initial direction and value, for example devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH) to hold the device in reset from the start.
In what order should probe power up a device before touching its registers, and what goes wrong if you get it wrong?Senior
Follow the hardware sequence in the datasheet, which is usually: enable supplies, wait for them to settle, enable clocks, release reset, wait for the reset time, then access registers.
ret = regulator_enable(vdd);
...
ret = clk_prepare_enable(clk);
...
reset_control_deassert(rst);
usleep_range(10, 20); /* datasheet reset release time */
id = readl(base + REG_ID);Getting it wrong is not a gentle failure. On many ARM SoCs a read from a peripheral whose clock is gated or whose power domain is off stalls the bus or raises an asynchronous external abort, which shows up as a hang or a confusing oops far from the cause. A device held in reset may read as all zeros, so an ID check fails. With runtime PM, the same sequence lives in the resume callback, and probe calls it through pm_runtime_resume_and_get(). A good habit is to read and check an ID or version register right after power-up and fail probe with a clear message if it is wrong.
User-space interfaces: file operations, ioctl, sysfs and mmap
These questions test whether your driver treats user space as untrusted and gives it an interface that will still make sense in five years.
Why must a driver use copy_to_user() instead of memcpy() or a plain pointer dereference?Junior
Because a user pointer is untrusted and may not point to valid, resident memory. copy_to_user() and copy_from_user() check that the range is inside the user part of the address space, so a malicious caller cannot pass a kernel address and read or overwrite kernel memory. They also handle page faults: if the page is swapped out or not yet allocated, the copy faults it in, and if the address is invalid the copy stops and returns the number of bytes it could not copy instead of crashing the kernel.
On modern CPUs there is a hardware layer too. SMAP on x86 and PAN on ARM make the kernel fault if it touches user memory outside the special accessor windows, so a plain dereference oopses even when the address is valid. Two consequences follow: the copy functions can sleep, so never call them under a spinlock, and a non-zero return should become -EFAULT.
Likely follow-up. What does copy_from_user() leave in the destination when it fails part-way?
How do you design an ioctl interface so it does not break later?Mid
Treat it as a permanent ABI and design it like one:
- Build command numbers with
_IO,_IOR,_IOWand_IOWR, which encode a type letter, number, direction and argument size, and check the letter is not already heavily used. - Use fixed-width types (
__u32,__u64) in structures, neverlongor pointers. Pass pointers as__u64so 32-bit and 64-bit callers see the same layout and nocompat_ioctltranslation is needed beyondcompat_ptr_ioctl. - Lay structures out with explicit padding so there are no hidden holes, and require reserved fields and flags to be zero so you can give them meaning later.
- Consider a size or version field for extensible structures.
- Return
-ENOTTYfor unknown commands,-EINVALfor bad arguments and-EFAULTfor bad pointers.
Also ask whether ioctl is right at all. A single tunable belongs in sysfs, and networking configuration usually belongs in netlink.
Likely follow-up. Why is -ENOTTY, not -EINVAL, the right error for an unknown command?
When would you use sysfs, debugfs, ioctl, configfs or netlink?Mid
Choose by who uses the interface and how stable it must be:
- sysfs: one value per file describing or controlling a device attribute, such as a threshold or a firmware version. It is ABI, so document it under
Documentation/ABI, and usesysfs_emit()for output. - debugfs: anything for developers, such as register dumps or fault injection. It has no ABI promise and is often not mounted on production systems, so never make a tool depend on it.
- ioctl: commands with structured arguments or performance needs on a device node, such as submitting a job to an accelerator.
- configfs: user space creating and configuring kernel objects, such as USB gadget functions.
- netlink: networking configuration and asynchronous events to many listeners.
A common review comment is "this binary blob in sysfs should be an ioctl" or the reverse. Also register sysfs attributes through dev_groups in the driver structure, so they exist before the uevent tells user space the device is there.
How do you implement poll() so that select and epoll work on your device?Mid
Keep a wait queue in the device and implement .poll like this:
static __poll_t mydev_poll(struct file *f, poll_table *pt)
{
struct mydev *d = f->private_data;
__poll_t mask = 0;
poll_wait(f, &d->wq, pt);
if (fifo_has_data(d))
mask |= EPOLLIN | EPOLLRDNORM;
if (fifo_has_space(d))
mask |= EPOLLOUT | EPOLLWRNORM;
return mask;
}poll_wait() does not sleep. It registers the caller on the wait queue so that a later wake_up_interruptible(&d->wq), typically from the interrupt or completion path when data arrives, makes the core call .poll again. The poll function must be cheap, must not block, and must recheck the real state each time. Return EPOLLERR or EPOLLHUP when the device has gone so waiters do not hang after removal.
How do you let user space mmap() device registers or a driver buffer?Senior
Implement .mmap and map physical pages into the caller's VMA, after checking the request against what you own.
For MMIO, the simplest safe call is vm_iomap_memory(vma, phys_base, size), which checks vm_pgoff and the length against the region and maps it with io_remap_pfn_range(). Set the page protection first, usually pgprot_noncached() for registers or pgprot_writecombine() for frame buffers. For a buffer from dma_alloc_coherent(), use dma_mmap_coherent(), which picks the right attributes for the platform. For ordinary kernel pages you can use remap_pfn_range() or vm_insert_page(), or provide a .fault handler that inserts pages on demand.
The pitfalls interviewers probe: never trust vm_pgoff and the VMA size without checking, never map more than whole pages you own (a partial page leaks neighbouring data), and think about what happens to the mapping when the device is removed. Exposing registers to user space also exposes DMA engines, which can then write anywhere unless an IOMMU confines the device.
What does a security review of an ioctl handler look for?Mid
The common findings are:
- Missing permission checks. Privileged operations need
capable(CAP_SYS_ADMIN)or a narrower capability, checked in the ioctl, not only by file mode. - Information leaks. Copying a stack structure to user space with uninitialised padding or unused fields leaks kernel memory. Zero it first with
= {}ormemset(). - Integer overflow.
count * sizeof(elem)can wrap. Usearray_size(),check_mul_overflow()orkmalloc_array(). - Unchecked indices. A user-supplied index used to read an array needs a bounds check, and on hot paths
array_index_nospec()to stop speculative out-of-bounds reads. - Double fetch and races. Copy once, and hold the right lock while you act on device state.
- Unbounded allocations. Cap sizes so a user cannot make the kernel allocate gigabytes.
Fuzzing with syzkaller, which knows ioctl argument formats through descriptions, finds many of these automatically.
Interrupts and deferred work
Interviewers spend a long time here because every driver bug about sleeping, latency or lost events traces back to execution context.
What may you not do in a hard interrupt handler, and why?Junior
A hard interrupt handler runs in atomic context: it interrupted whatever the CPU was doing, there is no process to put to sleep, and on the local CPU that interrupt line (and often all interrupts) is masked. So you must not:
- Sleep or call anything that might:
mutex_lock(),msleep(),copy_to_user(),kmalloc(GFP_KERNEL), most I2C and SPI transfers,wait_event(). - Take a lock that process context holds without disabling interrupts, or you deadlock on the same CPU.
- Spend long. Interrupt latency for everything else on that CPU grows with your handler time.
What a handler should do: read and acknowledge the device status so the line drops, capture the minimum data, and hand the rest to a threaded handler, workqueue or NAPI poll. It returns IRQ_HANDLED if its device raised the interrupt and IRQ_NONE otherwise.
Likely follow-up. Where would you put a 2 ms I2C read that the interrupt triggers?
When do you use request_threaded_irq(), and what does IRQF_ONESHOT do?Junior
Use a threaded interrupt when handling the event needs to sleep or takes long, which is almost always the case for devices on I2C or SPI and common for anything with a slow command path. You pass two functions: a primary handler that runs in hard interrupt context and a thread function that runs in a dedicated kernel thread with a real-time priority.
The primary handler returns IRQ_WAKE_THREAD to run the thread. If you pass NULL as the primary, the core uses a default that just wakes the thread. In that case you must set IRQF_ONESHOT, which keeps the interrupt line masked until the thread function returns. Without it, a level-triggered line from a device you have not yet serviced would fire again immediately and the CPU would never reach the thread. With a real primary handler you can avoid ONESHOT if that handler silences the device itself. devm_request_threaded_irq() is the managed form.
Compare softirqs, tasklets, workqueues and threaded interrupts as places to do deferred work.Mid
- Softirqs run in interrupt context after hard interrupts or in the
ksoftirqdthreads. The set is fixed at compile time (networking receive and transmit, block completion, timers, RCU). Drivers do not add new ones; NAPI is how network drivers use them. - Tasklets are built on softirqs. A given tasklet never runs on two CPUs at once, but it still cannot sleep. They are discouraged for new code because they give poor latency control.
- Workqueues run functions in kernel worker threads in process context, so they can sleep, take mutexes and do I2C transfers.
- Threaded interrupts are a dedicated thread per handler, tied to the interrupt, with the line masked while it runs if ONESHOT is set. They are the default answer for slow devices.
The decision rule: if you need to sleep, use a threaded IRQ or a workqueue; if you need high throughput with batching on a network or block path, use the subsystem mechanism (NAPI, blk-mq completion).
What is the difference between level-triggered and edge-triggered interrupts, and how can an edge interrupt be lost?Mid
A level-triggered line stays asserted as long as the device has an unserviced event, so the controller keeps signalling until the handler clears the cause. An edge-triggered line signals once on a transition. Level is safer for sharing and for not missing events, edge is simpler for things like a button or a pulse output.
The classic lost edge: the device raises its line for event A, the handler starts, reads the status, and while it is handling A the device finishes event B. If the device's output only produces a new edge when it goes from idle to active, and the line never dropped between A and B, no new edge arrives and B sits unhandled until something else happens. Fixes are to loop in the handler, re-reading status until it is clear before returning, to clear the status bits you have read before processing them, and to use the device's level mode if it has one. Interviewers also like the converse bug for level mode: forgetting to clear the cause, which causes an interrupt storm.
A board shows one CPU at 100% in interrupt context after your driver loads. How do you diagnose an interrupt storm?Senior
Start by finding which line is firing. Read /proc/interrupts twice a second apart: the count that is climbing by tens of thousands is the storm. /proc/irq/N/ and the handler name tell you whose it is.
Then the usual causes, in order of likelihood:
- The cause is never cleared. A level-triggered interrupt whose status bit the handler does not write back, or writes to the wrong register (write-one-to-clear versus write-zero), keeps the line asserted.
- Wrong trigger type in DT, such as level-high on a line that is active low, so the line looks asserted when idle.
- Clearing the status before the source. Some devices need the internal event source handled first, otherwise the status bit sets again at once.
- A threaded IRQ without ONESHOT on a level line.
- Returning IRQ_HANDLED on a shared line for an interrupt you did not handle.
Tools that help: ftrace with the irq:irq_handler_entry and irq_handler_exit events to see the rate and return values, and a quick printk of the status register with rate limiting.
Why does calling disable_irq() from your own interrupt handler deadlock, and how should remove() quiesce interrupts?Senior
disable_irq() masks the line and then waits, through synchronize_irq(), for any running instance of the handler to finish. Called from inside that handler, it waits for itself and never returns. Inside the handler use disable_irq_nosync(), which masks the line without waiting.
For remove, the safe order is:
- Tell the device to stop generating interrupts by clearing its interrupt enable register.
- Free the interrupt with
free_irq(), or callsynchronize_irq()if you keep it, which guarantees no handler is still running on another CPU. - Cancel deferred work that the handler might have queued, with the
_synccancel functions. - Only now free buffers and structures the handler used, and turn off clocks.
If you free the IRQ while the device can still assert a shared line, or free memory before the handler has finished on another CPU, you get use-after-free crashes that only show up under load.
Concurrency, locking and memory ordering
Locking questions separate candidates who memorised rules from those who can explain which contexts race and what each primitive guarantees.
When do you use a spinlock and when a mutex?Junior
Use a mutex when the critical section may sleep or is long, and every user is in process context. Use a spinlock when any user runs in atomic context (an interrupt handler, softirq or timer callback) or when the critical section is a few instructions.
The reasons: a mutex puts a waiting task to sleep, which cannot happen in interrupt context, and it is cheaper than spinning when the holder may sleep or take a long time. A spinlock busy-waits and disables preemption while held, so the holder must never sleep: if it did, another task on the same CPU could spin forever waiting for it. Spinlock critical sections should be short and contain no copy_to_user(), no allocation with GFP_KERNEL and no bus transfers on I2C or SPI.
A typical driver uses both: a mutex to serialise configuration from ioctl and sysfs, and a spinlock to protect the small state shared with the interrupt handler, such as a FIFO index.
When do you need spin_lock_irqsave() rather than spin_lock() or spin_lock_bh()?Mid
It depends on which contexts take the lock:
- Only process context:
spin_lock()is enough, or a mutex. - Process context and softirq or tasklet or timer: process context uses
spin_lock_bh(), which disables bottom halves on the local CPU so a softirq cannot interrupt the holder and spin on the same lock. The softirq itself uses plainspin_lock(). - Any context and a hard interrupt handler: the non-interrupt side uses
spin_lock_irqsave(), which disables local interrupts and saves the previous state. Otherwise the interrupt could arrive on the same CPU while the lock is held, and its handler would spin forever. Inside the hard handler plainspin_lock()is enough because the line is already masked.
spin_lock_irq() is a variant that disables and later enables interrupts unconditionally; it is only correct when you know interrupts were enabled on entry. Use irqsave when in doubt. Lockdep reports the wrong choice as an "inconsistent lock state" warning.
What is atomic context, and how do you find code that sleeps in it?Mid
Atomic context is any code path where the scheduler cannot be called: hard and soft interrupt handlers, code holding a spinlock, code with preemption or interrupts disabled, and RCU read-side critical sections for normal RCU. Sleeping there either deadlocks or corrupts scheduler state.
You cannot reliably test for it at run time. in_atomic() does not see a held spinlock on kernels built without preemption counting, which is why drivers should not change behaviour based on it. Instead, design so each function is documented to run in one context, and pass allocation flags down (GFP_ATOMIC versus GFP_KERNEL) when a helper serves both.
To find violations, enable CONFIG_DEBUG_ATOMIC_SLEEP. Functions that may sleep call might_sleep(), which then prints "BUG: sleeping function called from invalid context" with a stack trace the first time it happens, even if it would not have actually slept. Add might_sleep() to your own helpers that can block so the check covers them too. Lockdep and CONFIG_PROVE_LOCKING catch the related problems.
Explain RCU and when a driver would use it.Mid
Read-copy-update lets readers access shared data with almost no cost and no locks, while writers make changes by publishing a new version and freeing the old one only after every reader that might see it has finished.
Readers wrap access in rcu_read_lock() and rcu_read_unlock() and load pointers with rcu_dereference(). A writer, serialised against other writers by an ordinary lock, builds a new copy, publishes it with rcu_assign_pointer(), then waits for a grace period with synchronize_rcu() or defers the free with call_rcu() or kfree_rcu(). A grace period ends once every CPU has passed through a quiescent state, so every pre-existing reader is done.
It suits read-mostly data on hot paths: a table of registered handlers, a list of active devices looked up on every packet, a configuration pointer read in the interrupt handler. It is the wrong tool when writes are frequent or readers need to sleep (use SRCU or a lock). The common bug is touching the old object after publishing, or freeing it directly instead of after a grace period.
What is the difference between mb(), smp_mb(), dma_wmb() and READ_ONCE()?Senior
They order different things:
smp_mb(),smp_rmb(),smp_wmb()order memory accesses between CPUs. On a uniprocessor build they become compiler barriers only. Use them for lock-free data shared between CPUs, usually paired: a write barrier on the producer matched by a read barrier on the consumer.smp_store_release()andsmp_load_acquire()express the same thing more precisely.mb(),rmb(),wmb()are mandatory barriers that also order accesses against devices, including MMIO. Drivers rarely need them directly becausewritel()andreadl()already contain the needed ordering.dma_wmb()anddma_rmb()order accesses to coherent DMA memory as seen by a device. The classic use: write descriptor fields,dma_wmb(), then set the ownership bit; and read the ownership bit,dma_rmb(), then read the other fields.READ_ONCE()andWRITE_ONCE()stop the compiler from tearing, merging, repeating or caching an access. They give no CPU ordering on their own.
A strong candidate says which pair of accesses must stay in order, and to which observer, before picking a barrier.
Likely follow-up. Do you need a wmb() between filling a descriptor in coherent memory and the writel() to the doorbell?
When do you use a completion rather than a wait queue or a semaphore?Mid
A completion is for "wait until this one event has happened", which is the shape of most driver command paths: send a command to the device, wait for the interrupt that says it finished.
reinit_completion(&d->cmd_done);
writel(CMD_START, d->base + REG_CMD);
if (!wait_for_completion_timeout(&d->cmd_done, msecs_to_jiffies(100)))
return -ETIMEDOUT;The interrupt handler calls complete(&d->cmd_done). Completions handle the race where the event happens before the waiter sleeps, because the completion counts done events. A bare wait queue needs you to manage the condition variable yourself, which is right when the condition is something richer, such as "the FIFO has data or the device has gone". Semaphores are rarely appropriate in new driver code: as a lock a mutex is better, and as an event a completion is clearer.
Note the timeout. Hardware fails, and an untimed wait on a device that never answers creates an unkillable task. After a timeout, also make sure a late interrupt cannot complete a later command.
MMIO, DMA and memory
These questions test whether you understand what the CPU, caches, IOMMU and device each see, which is where the hardest driver bugs come from.
Why do drivers access registers with readl() and writel() instead of dereferencing a pointer?Junior
Register memory is not normal memory, and the accessors encode what the compiler and CPU must do with it:
- Mapping. You get a virtual address for registers with
ioremap()ordevm_platform_ioremap_resource(), which maps the physical range with device memory attributes (uncached, no speculative access). The result is avoid __iomem *, and the sparse checker (make C=1) warns if you dereference it directly. - Exactly one access of the right width. The compiler must not merge, split, cache or reorder register accesses. A 32-bit status register must be read with one 32-bit load.
- Ordering.
writel()contains a barrier so earlier writes to normal memory, such as DMA descriptors, are visible to the device before the register write.readl()completes before later memory reads. - Endianness.
readl()assumes a little-endian device and converts on big-endian CPUs. For big-endian registers useioread32be().
Port I/O on x86 uses inb() and outb(); ioread32() works for both mapped memory and ports.
What is the difference between writel() and writel_relaxed(), and why might a driver read a register right after writing it?Mid
The relaxed accessors still perform one access of the right size, in order with other accesses to the same device, but they drop the barrier that orders them against normal memory accesses. writel_relaxed() can become visible to the device before an earlier write to a DMA buffer, and readl_relaxed() can let a later read of DMA memory see stale data. They are useful in tight loops of register-only accesses, such as programming a block of configuration registers, where the barriers cost time and add nothing.
Reading a register after writing is a different issue: posted writes. On PCI and many SoC interconnects, a write can sit in a bridge buffer after the CPU has moved on. If the code needs the write to have reached the device before continuing, for example disabling interrupts before freeing the handler's data, or before a delay that the datasheet requires, a read from the same device flushes it, because the read cannot pass the earlier write on the way. The read is not about the value; it is about the ordering.
What is the difference between coherent and streaming DMA mappings?Mid
Coherent (consistent) memory comes from dma_alloc_coherent(). It returns a CPU virtual address and a dma_addr_t for the device, and both sides can access it at any time and see each other's writes without explicit cache maintenance. On systems without hardware coherency this is achieved by mapping the memory uncached, which makes CPU access slow. It suits long-lived shared structures: descriptor rings, command queues, status blocks.
Streaming mappings use dma_map_single(), dma_map_page() or dma_map_sg() on memory the driver already has, such as a network packet buffer. The mapping has a direction and an ownership rule: after mapping, the device owns the buffer, and the CPU must not touch it until dma_unmap_*() or dma_sync_*_for_cpu(). The calls do the cache cleaning or invalidating, or bouncing, that the platform needs. Streaming mappings keep normal cached memory for the CPU and are the right choice for data buffers that change for every transfer.
Always check dma_mapping_error() after a streaming map.
Explain the ownership rules for a streaming DMA buffer that is reused for many transfers.Mid
A mapped streaming buffer belongs either to the device or to the CPU, and the sync calls move it between them without unmapping:
dma = dma_map_single(dev, buf, len, DMA_FROM_DEVICE);
/* device owns buf: start the transfer, wait for completion */
dma_sync_single_for_cpu(dev, dma, len, DMA_FROM_DEVICE);
process(buf); /* CPU owns buf */
dma_sync_single_for_device(dev, dma, len, DMA_FROM_DEVICE);
/* device owns it again for the next transfer */On a non-coherent system, for_device with DMA_TO_DEVICE cleans (writes back) the CPU caches so the device reads what the CPU wrote, and for_cpu with DMA_FROM_DEVICE invalidates the caches so the CPU does not read stale lines. With bounce buffers, the same calls copy data. On a coherent x86 system they may do almost nothing, which is exactly why missing sync calls pass testing on x86 and corrupt data on ARM.
Use the most precise direction; DMA_BIDIRECTIONAL costs more maintenance. Never touch the buffer from the CPU while the device owns it, not even to read it.
What does an IOMMU change for a driver?Senior
For a correctly written driver, almost nothing in the code, and a lot in behaviour.
The IOMMU translates addresses the device emits (IO virtual addresses) to physical addresses using page tables the kernel controls. The DMA API returns an IOVA in dma_addr_t instead of a physical address. Consequences:
- Isolation. A device can only reach memory currently mapped for it. A buggy DMA engine faults instead of overwriting random kernel memory, and the IOMMU driver logs the faulting address, which is a powerful debugging aid.
- Mapping lifetime matters. Using a DMA address after unmap, which went unnoticed without an IOMMU, now faults or hits another buffer.
- Scatter-gather merging. Scattered pages can appear contiguous to the device.
- Cost. Map and unmap need page table updates and TLB invalidations, which shows up in high-rate drivers.
The rules for drivers follow from this: always use the DMA API, never virt_to_phys() for device addresses, and never assume a dma_addr_t equals a physical address.
How would you implement zero-copy DMA directly to and from a user-space buffer?Senior
Pin the user pages, build a scatter list, map it, and undo everything when the transfer completes:
- Validate the address and length, then call
pin_user_pages_fast()withFOLL_WRITEif the device will write. Pinning stops the pages from being migrated or swapped while the device uses them. - Build a table with
sg_alloc_table_from_pages(), which merges physically adjacent pages. - Map with
dma_map_sgtable(), check the result and program the device with each mapped segment, respecting its maximum segment size. - On completion,
dma_unmap_sgtable(), thenunpin_user_pages_dirty_lock()for pages the device wrote so the kernel knows they changed, and free the table.
Points that matter in an interview: the user buffer is usually not page aligned, so the first and last pages are partial, and the process can exit or unmap the range while DMA is running, so the pin, not the user mapping, is what keeps the pages alive. For buffers shared between devices, such as a camera and a GPU, the kernel's dma-buf framework is the standard alternative.
Bus subsystems: PCI, USB, networking, block, I2C and SPI
Teams hire for a device class, so expect detailed questions on the subsystem you claim and solid basics on the others.
Walk through the probe function of a PCIe driver.Mid
A typical order, using managed helpers where they exist:
pcim_enable_device()to wake the device and enable memory decoding.- Request and map the BARs you use, with
pci_request_regions()andpci_iomap()or their managed equivalents, so no other driver claims the same resources. pci_set_master()so the device may initiate DMA.- Set the DMA mask with
dma_set_mask_and_coherent(). - Read and check a version or capability register, and reset the device to a known state.
- Allocate interrupt vectors with
pci_alloc_irq_vectors()and request handlers. - Allocate rings and other DMA memory.
- Register with the subsystem last:
register_netdev(), a block disk, a char device. From this moment requests can arrive.
remove() does the reverse: unregister from the subsystem first, so no new requests come in, then quiesce the device, free interrupts, free DMA memory and disable the device. Interviewers watch for registration too early and for DMA being enabled before the rings exist.
Explain USB endpoints, transfer types and URBs from a driver's point of view.Mid
A USB device exposes interfaces, and each interface has endpoints, which are one-directional pipes identified by number and direction. Endpoint 0 is the bidirectional control pipe every device has. The other endpoints have one of four transfer types:
- Control for setup and configuration requests.
- Bulk for large data with guaranteed delivery but no timing guarantee, such as storage.
- Interrupt for small, latency-bounded polling, such as a keyboard or a status endpoint.
- Isochronous for streaming with reserved bandwidth and no retries, such as audio and video.
A Linux USB driver binds to an interface, not a device. It moves data with URBs: allocate with usb_alloc_urb(), fill with usb_fill_bulk_urb() and friends, giving a buffer and a completion callback, and submit with usb_submit_urb(). The completion runs in atomic context and reports status in urb->status, where -ENOENT, -ECONNRESET and -ESHUTDOWN mean "cancelled" or "device gone" rather than errors to retry. In disconnect(), kill outstanding URBs with usb_kill_urb() or an anchor before freeing anything.
Describe the transmit path of a network driver and how it applies flow control.Mid
The stack calls ndo_start_xmit(skb, netdev) with the queue lock held and bottom halves disabled, so it must not sleep. The driver:
- Maps the packet's head and fragments for DMA (
dma_map_single()andskb_frag_dma_map()), checking for mapping errors. - Fills one or more TX descriptors, sets the ownership bit last after
dma_wmb(), and records the skb so it can be freed later. - Writes the tail or doorbell register.
- If the ring no longer has room for a maximum-size packet, calls
netif_stop_queue()(or the per-queue variant) so the stack stops sending. - Returns
NETDEV_TX_OK.
Completion happens later, usually in NAPI poll: the driver walks the ring, unmaps buffers, frees skbs with napi_consume_skb(), updates statistics, and when enough space is free calls netif_wake_queue(). Returning NETDEV_TX_BUSY is treated as a driver bug in most cases; stopping the queue early avoids it.
What is NAPI, and why does it exist?Senior
NAPI is the network stack's interrupt mitigation scheme. A purely interrupt-driven receive path takes one interrupt per packet, and at high packet rates the CPU spends all its time entering and leaving interrupt handlers and never gets to process the packets, a condition called receive livelock.
With NAPI, the receive interrupt handler disables further receive interrupts from the device and calls napi_schedule(). The poll function then runs in softirq context with a budget, typically 64 packets. It processes up to that many completed receive descriptors, hands packets up with napi_gro_receive(), and refills the ring. If it used less than the budget, the ring is drained: it calls napi_complete_done() and re-enables the device interrupt. If it used the whole budget, it returns without re-enabling, and the core polls it again later, sharing the CPU fairly with other devices.
So under light load you get interrupt latency, and under heavy load you get polling efficiency. A common bug is re-enabling interrupts before napi_complete_done() returns true, which can lose a wake-up.
How do I2C and SPI client drivers differ from memory-mapped drivers, and what does regmap give you?Mid
Every register access on I2C or SPI is a bus transaction through a controller driver, and those transfers sleep. So register access is slow (tens to hundreds of microseconds), can fail with an error code, and is forbidden in atomic context. Interrupt handling therefore uses a threaded IRQ, and anything polled uses delayed work.
regmap is an abstraction over register maps that most such drivers use. You describe the register width, value width and bus in a regmap_config, create it with devm_regmap_init_i2c() or the SPI equivalent, then use regmap_read(), regmap_write() and regmap_update_bits(). It gives:
- Locking around read-modify-write sequences.
- An optional register cache, which saves bus traffic and lets you restore all registers after a power loss with
regcache_sync(). - Volatile, readable and writeable register lists, so mistakes are caught.
- Debugfs dumps of register contents, and the same driver code for I2C and SPI variants of a chip.
There is also regmap_irq for chips with an interrupt status and mask register block.
How does a blk-mq block driver receive and complete requests?Senior
The driver allocates a tag set that describes its hardware queues, the depth of each and the size of a per-request private area, with blk_mq_alloc_tag_set(). From it, it creates a disk with its request queue. The block layer hands requests to the driver's queue_rq() callback.
In queue_rq() the driver calls blk_mq_start_request(), translates the request into device commands, maps its data with blk_rq_map_sg() and the DMA API, uses the request's tag as the command identifier, and rings the doorbell. If the device has no room it returns BLK_STS_RESOURCE (or BLK_STS_DEV_RESOURCE) and the core retries later; any other non-OK status fails the request.
On completion, usually from an interrupt, the driver looks up the request from the tag with blk_mq_tag_to_rq(), records the status and calls blk_mq_complete_request(), which can redirect completion to the submitting CPU, and the complete callback ends it with blk_mq_end_request(). A timeout callback handles commands the device never finished. NVMe is the reference design for this model.
Power management
Power bugs appear only after suspend cycles or idle periods, so interviewers test whether you can reason about state that disappears while the driver is not looking.
What is the difference between system sleep and runtime power management?Junior
System sleep (suspend to RAM, suspend to idle, hibernation) is a whole-system transition started by user space or a policy daemon. The PM core freezes user space, then calls every driver's suspend callbacks in an order derived from the device hierarchy and device links, and later the resume callbacks in reverse. The driver must stop all activity and save whatever state will be lost.
Runtime PM is per device and happens while the system is running. A driver powers its device down when it is idle and up again when it is needed, through runtime_suspend and runtime_resume callbacks. The PM core tracks a usage count and calls the callbacks when the count reaches zero, optionally after an autosuspend delay so that bursts of activity do not cause constant power cycling.
Many drivers implement system sleep by reusing runtime PM, with pm_runtime_force_suspend() and pm_runtime_force_resume().
Show how a driver uses the runtime PM API around a register access.Mid
In probe, enable runtime PM with autosuspend, then in each I/O path take a reference while you use the hardware:
/* probe */
pm_runtime_set_autosuspend_delay(dev, 200);
pm_runtime_use_autosuspend(dev);
devm_pm_runtime_enable(dev);
/* I/O path */
ret = pm_runtime_resume_and_get(dev);
if (ret)
return ret;
val = readl(priv->base + REG_DATA);
pm_runtime_mark_last_busy(dev);
pm_runtime_put_autosuspend(dev);The trap interviewers look for: pm_runtime_get_sync() increments the usage count even when resume fails, so code that returns on error without calling pm_runtime_put_noidle() leaks a reference and the device never suspends again. pm_runtime_resume_and_get() drops the count itself on failure, which is why it is preferred. Every get must have exactly one put on every path, including error paths.
What should a driver's suspend and resume callbacks do?Mid
Suspend must leave the device quiet and its state recoverable:
- Stop accepting new work and wait for or cancel work in flight, including timers and work items.
- Disable the device's interrupts at the device so it does not fire during the transition.
- Save any register state that will be lost, unless a regmap cache holds it.
- Configure wake-up if the device is a wake source, and select the sleep pin state.
- Gate clocks and disable regulators in the order the hardware needs.
Resume reverses it: power, clocks, reset if required, restore registers (often regcache_sync()), re-enable interrupts and restart queues. Resume must assume the device lost all state, because on many platforms it did, and must handle the device not coming back by failing cleanly rather than hanging.
The PM core offers phases: prepare, suspend, suspend_late, suspend_noirq and the matching resume phases.
How does a device wake the system from sleep, and what must the driver do?Senior
A wake source is an interrupt that stays armed while the system sleeps and brings it back, such as a power button, an RTC alarm, a network magic packet or a modem.
The driver's part:
- Declare the capability with
device_init_wakeup(dev, true)in probe if the hardware and board support it, often from awakeup-sourceDT property. This creates thepower/wakeupfile that lets user space enable or disable wake-up. - In suspend, if
device_may_wakeup(dev)is true, configure the device to signal on the wake event and callenable_irq_wake(irq)so the interrupt controller keeps the line armed. Disable everything else. - In resume, call
disable_irq_wake(irq)and restore normal operation. - When the wake interrupt arrives, call
pm_wakeup_event()or hold a wakeup source while processing the event, so the system does not go back to sleep before user space has handled it.
The usual bugs are unbalanced enable_irq_wake() calls and wake events lost because resume cleared the status without reporting it.
Which clock and regulator calls may sleep, and why does that matter for runtime PM?Mid
The clock API is split into two halves. clk_prepare() and clk_unprepare() may sleep, because preparing a clock can involve starting a PLL or talking to a power management IC over I2C. clk_enable() and clk_disable() must not sleep and can be called in atomic context. clk_prepare_enable() does both and so may sleep. Regulator calls generally may sleep.
That matters because runtime PM callbacks normally run in process context, where sleeping is fine, but some drivers mark themselves pm_runtime_irq_safe() so they can resume the device from an interrupt handler. Then the callbacks run with interrupts disabled and may only use non-sleeping operations: keep clocks prepared from probe and only enable and disable them in the callbacks. The cost is that the parent device is kept permanently powered, so IRQ-safe runtime PM should be a deliberate exception. A strong candidate also mentions that power domains, attached through the PM domain framework, handle much of the switching without the driver calling the regulator directly.
How do you protect an interrupt handler from touching a runtime-suspended device?Senior
On a shared interrupt line, or if a wake-up interrupt fires during the resume transition, the handler can run while your device's clocks are off. Reading a register then returns garbage or stalls the bus.
Options, depending on the design:
- In the handler, use
pm_runtime_get_if_active()(orpm_runtime_get_if_in_use()), which takes a reference only if the device is already active. If it fails, returnIRQ_NONEbecause your device could not have raised the interrupt while suspended. - Keep a driver flag, set in
runtime_suspendafter disabling the device's interrupt output and cleared inruntime_resume, protected by the same spinlock the handler takes. - Disable the interrupt with
disable_irq()inruntime_suspend, if the line is not shared, and enable it inruntime_resume.
Whatever you choose, the device's own interrupt enable bits must be cleared before power goes off, and pending status must be cleared on resume before enabling, or you get a spurious interrupt at a bad moment.
Debugging, testing and upstreaming
Senior interviews weigh how you find and prove a bug, and whether your patches would survive review on a kernel mailing list.
How do you log from a driver, and how do you turn debug messages on in the field?Junior
Use the device-aware helpers: dev_err(), dev_warn(), dev_info() and dev_dbg(). They prefix the driver and device name, so logs from several instances can be told apart. Use pr_ variants only where there is no device. Error messages should say what failed and include the error code, and should not fire for normal conditions such as deferral.
dev_dbg() compiles to nothing unless DEBUG is defined, or, more usefully, CONFIG_DYNAMIC_DEBUG is enabled. Then each call site can be switched on at run time:
echo 'module mydrv +p' > /sys/kernel/debug/dynamic_debug/control
echo 'file mydrv_core.c line 210 +p' > /sys/kernel/debug/dynamic_debug/controlIt can also be enabled at boot with dyndbg= parameters. For messages that could repeat at interrupt rate, use dev_err_ratelimited() or dev_warn_once(), since a flood of console output can itself cause timing problems and lockups on slow serial consoles. trace_printk() is a faster option during development because it writes into the ftrace buffer instead of the console.
You are given a kernel oops from your driver. How do you find the faulting line?Mid
Read the header first: the type of fault (NULL pointer dereference, paging request at a given address, BUG at file:line), the CPU and process, and the Tainted flags, which show whether out-of-tree or previously warned code is involved. Then the program counter, printed as function+offset/length, and the call trace.
To get a source line, you need the exact module binary with debug information:
./scripts/faddr2line drivers/foo/mydrv.ko mydrv_irq+0x4c/0x120
./scripts/decode_stacktrace.sh vmlinux . drivers/foo < oops.txtOr use gdb with list *(mydrv_irq+0x4c). Then reason from the register dump: a faulting address of 0x0000000000000010 means a NULL structure pointer with the accessed field at offset 0x10, which usually identifies the field and therefore the pointer. Address patterns such as 0x6b6b6b6b (slab poison) point to a use-after-free. Finally ask what context the code ran in and what else could have freed or changed the object, which is where the real cause usually is.
How do you use ftrace to understand what a driver is doing?Mid
ftrace is the kernel's built-in tracer, controlled through /sys/kernel/tracing or the trace-cmd front end. The parts drivers use most:
- Function and function_graph tracers. Set
set_ftrace_filterto your driver's functions and see every call with timing. function_graph shows nesting and duration, which is the fastest way to see where probe or resume spends its time. - Tracepoints. Events such as
irq:irq_handler_entry,workqueue:workqueue_execute_start,i2c:i2c_writeorblock:block_rq_issueshow what subsystems are doing without changing code. Drivers can define their own withTRACE_EVENT(). - trace_printk(). A cheap printf into the trace buffer that does not disturb timing the way printk to a console does.
- Latency tracers such as
irqsoffandpreemptoff, which find the longest sections with interrupts or preemption disabled.
Because trace data goes into a per-CPU ring buffer, it captures interleavings across CPUs, so it is the right tool for races between an interrupt and process context.
Which kernel debug options would you enable when developing a driver, and what does each catch?Mid
A debug configuration for driver work usually includes:
- KASAN: out-of-bounds accesses and use-after-free on heap, stack and globals, with allocation and free stack traces. Large overhead, so development only.
- KFENCE: a low-overhead sampling version of the same idea, suitable for production.
- Lockdep and PROVE_LOCKING: lock order and context bugs before they deadlock.
- DEBUG_ATOMIC_SLEEP: sleeping in atomic context.
- KCSAN: data races between CPUs on plain accesses.
- UBSAN: undefined behaviour such as shifts too large and signed overflow.
- kmemleak: memory that is no longer referenced, reported through debugfs.
- DMA_API_DEBUG: unmapping with the wrong size or direction, CPU access to buffers the device owns, leaked mappings at unbind.
- DEBUG_OBJECTS for timers and work, and SLUB debugging with poisoning.
Pair these with fuzzing through syzkaller for the user interface and repeated bind and unbind or module load and unload loops for lifetime bugs. A candidate who uses DMA API debugging stands out, because few people know it exists.
The system hangs during a stress test of your driver. How do you find out where?Senior
First decide what kind of hang it is, because the tools differ:
- One CPU stuck with interrupts disabled (spinning on a lock with interrupts off, a loop in a hard interrupt handler): the hard lockup detector, driven by the NMI watchdog, prints a backtrace. Enable it and make sure the console reaches you, through a serial port or netconsole.
- A CPU stuck in the kernel without scheduling: the soft lockup detector reports it after about twenty seconds.
- Tasks blocked forever in uninterruptible sleep, typically waiting on a mutex or a completion: the hung task detector reports them with stacks after a timeout, and SysRq
wdumps all blocked tasks on demand. SysRqlshows backtraces on all CPUs. - Completely dead machine: configure kdump so a panic, which you can force with the
panic_on_sysctls or SysRqc, boots a capture kernel that saves memory. Then use thecrashtool to inspect stacks, locks and your driver's structures.
Then make it reproducible and smaller: lockdep often explains a deadlock that the watchdogs only detect.
Describe how you would get a new driver accepted upstream.Mid
Prepare a clean patch series against the maintainer's tree for that subsystem, not an old stable release:
- Split the work into logical, bisectable patches: device tree binding first, then the driver, then the DTS changes, each building on its own.
- Run
scripts/checkpatch.pl --strict, build withW=1andC=1for sparse, and for bindings runmake dt_binding_check. - Write commit messages that say why the change is needed, not only what it does, and add a
Signed-off-by:line, which certifies the Developer Certificate of Origin. - Find recipients with
scripts/get_maintainer.pland send plain-text patches withgit send-email, with a cover letter for a series. - Respond to every review comment, either by changing the code or explaining why not. Send the next version with a
v2prefix and a changelog below the---line.
Patience and courtesy matter as much as code. Maintainers are busy; a polite reminder after a couple of weeks is normal, while resending unchanged or arguing every point is not.
Windows and RTOS drivers
Many driver roles touch a second operating system, and interviewers check that your concepts transfer rather than only your Linux API knowledge.
What is IRQL in Windows, and what can driver code do at each level?Mid
IRQL (interrupt request level) is the Windows kernel's priority scheme for the processor. Code at a given IRQL can only be interrupted by something at a higher level. The levels a driver meets are:
- PASSIVE_LEVEL: normal thread context. You can wait, take any lock, touch pageable memory and call almost any routine.
- APC_LEVEL: asynchronous procedure calls are masked, used inside some locks and paging I/O.
- DISPATCH_LEVEL: the scheduler cannot run on this CPU. Deferred procedure calls run here, and spin locks raise to it. You cannot wait on objects with a non-zero timeout and must not touch pageable code or data, because a page fault cannot be serviced.
- DIRQL: device interrupt levels, where the interrupt service routine runs.
The rule maps closely to Linux: DISPATCH_LEVEL behaves like atomic context and PASSIVE_LEVEL like process context. Breaking it gives the well-known IRQL_NOT_LESS_OR_EQUAL bug check. Driver Verifier, run during development, makes these errors show up reliably.
Compare WDM, KMDF and UMDF.Mid
WDM is the original Windows Driver Model. Drivers handle I/O request packets and Plug and Play and power IRPs directly, and getting the PnP and power state machines and cancellation right is famously difficult.
KMDF, part of the Windows Driver Frameworks, runs in the kernel on top of WDM. It provides objects with reference counting and parent-child lifetimes, queues that handle request flow and cancellation, and default PnP and power handling, so a driver writes event callbacks such as EvtDevicePrepareHardware and EvtIoRead instead of state machines. Most new kernel-mode drivers use KMDF.
UMDF version 2 uses nearly the same programming model but runs the driver in a user-mode host process. A crash takes down the host, not the system, and debugging is simpler, at the cost of extra latency and no direct access to some kernel facilities. It suits USB peripherals, sensors and other devices without hard latency needs. The interview point is choosing the least privileged model that meets the device's needs, in the same way a Linux engineer might consider a user-space driver.
How do the ISR and DPC model in Windows compare with Linux interrupt handling?Mid
They have the same shape. The Windows interrupt service routine runs at device IRQL, checks whether its device interrupted, silences it, saves the minimum state and returns TRUE or FALSE, like IRQ_HANDLED or IRQ_NONE. It then queues a deferred procedure call, which runs later at DISPATCH_LEVEL with device interrupts enabled. That corresponds to a Linux tasklet or softirq: atomic, not allowed to sleep, used to complete requests.
Work that must wait or call routines that only work at PASSIVE_LEVEL goes to a work item, the counterpart of a Linux workqueue, or KMDF's passive-level interrupt handling, which is close in spirit to a Linux threaded interrupt.
In KMDF, the callbacks are EvtInterruptIsr and EvtInterruptDpc, and the ISR queues the DPC with WdfInterruptQueueDpcForIsr(). Data shared between the ISR and other code is protected by the interrupt lock (WdfInterruptAcquireLock()), the counterpart of spin_lock_irqsave(). If you can explain this mapping, you show that you understand the concepts rather than one API.
How does the Zephyr device driver model work?Mid
Zephyr builds the device list at compile time from device tree. The build turns the DTS into generated C macros, and a driver uses them instead of parsing anything at run time. A driver sets DT_DRV_COMPAT to its compatible string, defines a config structure (constant data such as base address and IRQ from DT_INST_REG_ADDR() and similar macros) and a data structure (run-time state), and instantiates one device per enabled node with DEVICE_DT_INST_DEFINE() inside DT_INST_FOREACH_STATUS_OKAY().
Each device has an init function run at a chosen level (PRE_KERNEL_1, PRE_KERNEL_2, POST_KERNEL, APPLICATION) with a priority, which orders dependencies. Device classes define an API structure of function pointers, such as the I2C or sensor API, and applications call the class API, which dispatches to the driver. Applications get a device with DEVICE_DT_GET() and must check device_is_ready().
Compared with Linux there is no probe, no hot plug and no dynamic binding: everything is resolved at build time, which saves RAM and flash but means the image is built for one board description.
On an RTOS such as FreeRTOS or Zephyr, how do you pass an event from an interrupt handler to a driver task?Mid
Keep the ISR short and signal a task with an ISR-safe primitive. In FreeRTOS, only the FromISR API variants may be called from an interrupt, and they report whether a higher priority task was woken so you can request a context switch on exit:
void UART_IRQHandler(void)
{
BaseType_t woken = pdFALSE;
uint8_t b = UART->DR; /* reading clears the flag */
xQueueSendFromISR(rx_queue, &b, &woken);
portYIELD_FROM_ISR(woken);
}For a single "something happened" event, a direct-to-task notification is lighter than a queue or semaphore. In Zephyr, ISRs can call k_sem_give(), k_msgq_put() with K_NO_WAIT, or submit a k_work item to a work queue.
Also cover interrupt priorities on Cortex-M: in FreeRTOS, ISRs that call the kernel API must run at or below configMAX_SYSCALL_INTERRUPT_PRIORITY, otherwise they can corrupt kernel state. ISRs above that level are allowed for very low latency work but cannot use any RTOS calls.
You are asked to port a Linux SPI sensor driver to Zephyr. What changes and what stays the same?Senior
The device knowledge stays the same: register map, power-up sequence, conversion formulas, quirks and errata. That is usually most of the value in the old driver, so I would first separate it into plain functions that take a "read register" and "write register" callback.
What changes:
- Binding and resources. Probe disappears. Configuration comes from devicetree macros at build time, and the SPI bus and chip select come from
SPI_DT_SPEC_INST_GET(). - Bus access.
spi_write_dt()andspi_transceive_dt()replacespi_sync()or regmap. - Interrupts. The data-ready GPIO uses
gpio_pin_interrupt_configure_dt()and a callback that gives a semaphore or submits work, instead of a threaded IRQ. - Subsystem API. The Linux IIO interface becomes the Zephyr sensor API,
sample_fetchandchannel_get, with values instruct sensor_value. - Memory and power. No dynamic allocation; state is in per-instance data structures. Power management uses Zephyr's device PM hooks.
Licensing also matters: Linux drivers are GPL and Zephyr is Apache 2.0, so copying code across needs the copyright holder's agreement.
Coding exercises
Set a timer for 30 minutes per exercise. Write the code in an editor, compile it, and test the edge cases before you open the solution.
- Register field helpers and a safe read-modify-writeEasy · C
A control register has an enable bit at bit 0, a 4-bit MODE field at bits 7:4 and an 8-bit DIV field at bits 23:16. Other bits belong to other functions and must not change. Write
GENMASK, helpers to pack and extract a field from a mask, andint ctrl_configure(volatile uint32_t *ctrl, uint32_t mode, uint32_t div)that sets both fields and the enable bit in one read and one write. Return-EINVALif a value does not fit its field.Example: with the register at
0xFF00000A,ctrl_configure(r, 5, 200)leaves it at0xFFC8005B.Hints
- Build the mask from two shifted all-ones values so GENMASK(31, 0) does not shift by 32.
- The lowest set bit of the mask gives the shift, so the mask alone is enough to pack a field.
- Check the range before shifting; a value too large would silently spill into neighbouring bits.
Solution and explanation
#include <errno.h> #include <stdint.h> #define BIT(n) (1u << (n)) #define GENMASK(h, l) ((~0u >> (31 - (h))) & (~0u << (l))) #define CTRL_EN BIT(0) #define CTRL_MODE GENMASK(7, 4) #define CTRL_DIV GENMASK(23, 16) /* mask must be a non-zero contiguous run of bits */ static inline uint32_t field_max(uint32_t mask) { return mask >> __builtin_ctz(mask); } static inline uint32_t field_prep(uint32_t mask, uint32_t val) { return (val << __builtin_ctz(mask)) & mask; } static inline uint32_t field_get(uint32_t mask, uint32_t reg) { return (reg & mask) >> __builtin_ctz(mask); } /* Change only the bits in mask; one read and one write of the register. */ static inline void reg_update_bits(volatile uint32_t *reg, uint32_t mask, uint32_t val) { uint32_t v = *reg; v = (v & ~mask) | (val & mask); *reg = v; } int ctrl_configure(volatile uint32_t *ctrl, uint32_t mode, uint32_t div) { if (mode > field_max(CTRL_MODE) || div > field_max(CTRL_DIV)) return -EINVAL; reg_update_bits(ctrl, CTRL_MODE | CTRL_DIV | CTRL_EN, field_prep(CTRL_MODE, mode) | field_prep(CTRL_DIV, div) | CTRL_EN); return 0; }GENMASK(h, l)shifts all ones right by31 - hand left byland ANDs the two, which never shifts a 32-bit value by 32 (undefined behaviour in C). The field helpers take the shift from__builtin_ctz(mask), so a field is described by its mask only, the same idea as the kernel'sFIELD_PREP()andFIELD_GET().reg_update_bits()reads once, changes only masked bits and writes once. It is not atomic: if an interrupt handler or another CPU also writes this register, the read-modify-write must be inside a spinlock, or bits set by the other path are lost. In a real driver the access would go throughreadl()andwritel()on an__iomempointer rather than avolatilepointer.Interviewers listen for range checking before packing, for the observation about atomicity, and for awareness of write-one-to-clear bits: if any bit in the register is W1C, writing back the read value clears it accidentally, so such registers need a different pattern.
- Parse a USB device descriptor from untrusted bytesEasy · C
A USB device returns an 18-byte device descriptor. Multi-byte fields are little-endian:
bcdUSBat offset 2,idVendorat 8,idProductat 10 andbcdDeviceat 12.bDeviceClassis at 4,bMaxPacketSize0at 7 andbNumConfigurationsat 17. Writeparse_device_desc(const uint8_t *buf, size_t len, struct usb_dev_info *out)that validates and decodes it on any host endianness. Reject short buffers, a wrongbLengthor type, an invalid endpoint 0 packet size (8, 16, 32 or 64, or 9 meaning 512 bytes for USB 3.x), and zero configurations.Hints
- Never cast the buffer to a packed struct pointer; build each value from bytes.
- Check the buffer length before reading any field.
Solution and explanation
#include <errno.h> #include <stddef.h> #include <stdint.h> struct usb_dev_info { uint16_t bcd_usb, vendor, product, bcd_device; uint8_t dev_class, max_packet0, num_configs; }; static uint16_t get_le16(const uint8_t *p) { return (uint16_t)(p[0] | (p[1] << 8)); } /* Parse an 18-byte USB device descriptor from an untrusted buffer. */ int parse_device_desc(const uint8_t *buf, size_t len, struct usb_dev_info *out) { if (!buf || !out || len < 18) return -EINVAL; if (buf[0] != 18 || buf[1] != 0x01) /* bLength, bDescriptorType DEVICE */ return -EINVAL; out->bcd_usb = get_le16(buf + 2); out->dev_class = buf[4]; out->max_packet0 = buf[7]; out->vendor = get_le16(buf + 8); out->product = get_le16(buf + 10); out->bcd_device = get_le16(buf + 12); out->num_configs = buf[17]; switch (out->max_packet0) { case 8: case 16: case 32: case 64: break; case 9: /* USB 3.x: exponent, 2^9 = 512 bytes */ if (out->bcd_usb >= 0x0300) break; return -EINVAL; default: return -EINVAL; } if (out->num_configs == 0) return -EINVAL; return 0; }The function treats every byte as hostile, which is the right attitude for anything a device sends: a faulty or malicious USB device can return any length and any values. It checks the caller's buffer length first, then the descriptor's own length and type, then the semantic constraints.
Values are assembled from bytes with
get_le16(), which is correct on big-endian and little-endian hosts and avoids unaligned access. Casting the buffer to astructpointer would depend on the compiler's padding and the CPU's alignment rules, and on big-endian hosts would silently byte-swap the IDs. In the kernel you would useget_unaligned_le16()or__le16fields withle16_to_cpu(), whichsparsechecks.The
bMaxPacketSize0rule shows the follow-up interviewers like: the same field changes meaning with the USB version (a byte count before USB 3.0, an exponent after), so validation depends onbcdUSB. In a real class driver the USB core has already validated this descriptor; the same discipline applies to class and vendor descriptors it does not understand. - Lock-free single-producer single-consumer ring bufferMedium · C
An interrupt handler produces 32-bit samples and a reader thread consumes them. Implement a fixed-size ring with
bool ring_push(struct spsc_ring *r, uint32_t v)andbool ring_pop(struct spsc_ring *r, uint32_t *v)that is safe with exactly one producer and one consumer running concurrently on different CPUs, without locks. The capacity must be the full 1024 slots (do not waste a slot), and the indices must survive wrapping of a 32-bit counter. Use C11 atomics.Hints
- Let head and tail count up freely and mask them only when indexing; head minus tail is the fill level even after wrap-around.
- The producer must publish the slot before the new head; the consumer must finish reading the slot before the new tail.
Solution and explanation
#include <stdatomic.h> #include <stdbool.h> #include <stdint.h> #define RING_SIZE 1024u /* power of two */ #define RING_MASK (RING_SIZE - 1u) struct spsc_ring { _Atomic uint32_t head; /* written only by the producer */ _Atomic uint32_t tail; /* written only by the consumer */ uint32_t slot[RING_SIZE]; }; /* Producer side. head and tail run freely; unsigned wrap keeps head - tail correct. */ bool ring_push(struct spsc_ring *r, uint32_t v) { uint32_t head = atomic_load_explicit(&r->head, memory_order_relaxed); uint32_t tail = atomic_load_explicit(&r->tail, memory_order_acquire); if (head - tail == RING_SIZE) return false; /* full */ r->slot[head & RING_MASK] = v; /* release: the slot write is visible before the new head */ atomic_store_explicit(&r->head, head + 1, memory_order_release); return true; } /* Consumer side. */ bool ring_pop(struct spsc_ring *r, uint32_t *v) { uint32_t tail = atomic_load_explicit(&r->tail, memory_order_relaxed); uint32_t head = atomic_load_explicit(&r->head, memory_order_acquire); if (head == tail) return false; /* empty */ *v = r->slot[tail & RING_MASK]; /* release: we finished reading the slot before the producer may reuse it */ atomic_store_explicit(&r->tail, tail + 1, memory_order_release); return true; }Each index has exactly one writer: the producer owns
head, the consumer ownstail. That is why no lock or compare-and-swap is needed. The counters run freely and are masked withRING_SIZE - 1when indexing, so unsigned subtractionhead - tailgives the number of items even after the counters wrap pastUINT32_MAX, and all 1024 slots are usable. This requires a power-of-two size.Ordering is the heart of the answer. The producer writes the slot, then stores
headwith release semantics, so a consumer that loadsheadwith acquire is guaranteed to see the slot contents. Symmetrically, the consumer reads the slot, then releasestail, and the producer acquirestailbefore overwriting a slot, so it never overwrites data still being read. The test harness runs five million items across two threads and passes under ThreadSanitizer.In the kernel the same structure is
kfifoorcirc_bufwithsmp_load_acquire()andsmp_store_release(). Interviewers ask what breaks with two producers (both can claim the same slot), and whether this works between a CPU and a DMA engine (no: the device needsdma_wmb()and coherent memory, the subject of a later exercise). - Validate a variable-length ioctl payloadMedium · C
An ioctl receives a configuration blob that the driver has already copied into kernel memory: a header
{u32 version; u32 count}followed bycountrecords, each{u16 type; u16 len}and thenlendata bytes padded to a multiple of 4. Writeint cfg_parse(const uint8_t *buf, size_t size, rec_fn fn, void *ctx)that callsfnfor each record and returns 0 only if the whole blob is well formed: version 1, at most 64 records, every record inside the buffer, and no trailing bytes. Every length in the blob is controlled by an attacker. Errors fromfnare returned as they are.Hints
- Compare lengths as size minus offset, never as offset plus length, so nothing can overflow.
- Read headers with memcpy so alignment does not matter.
Solution and explanation
#include <errno.h> #include <stddef.h> #include <stdint.h> #include <string.h> #define CFG_VERSION 1u #define CFG_MAX_RECORDS 64u struct cfg_hdr { uint32_t version; uint32_t count; }; /* then count records */ struct rec_hdr { uint16_t type; uint16_t len; }; /* then len bytes, padded to 4 */ typedef int (*rec_fn)(uint16_t type, const uint8_t *data, uint16_t len, void *ctx); /* * buf/size is a kernel copy of the user payload (copied once, so the user cannot * change it between the check and the use). Every length is untrusted. */ int cfg_parse(const uint8_t *buf, size_t size, rec_fn fn, void *ctx) { struct cfg_hdr h; size_t off = sizeof(h); uint32_t i; if (size < sizeof(h)) return -EINVAL; memcpy(&h, buf, sizeof(h)); /* no unaligned or aliasing tricks */ if (h.version != CFG_VERSION || h.count > CFG_MAX_RECORDS) return -EINVAL; for (i = 0; i < h.count; i++) { struct rec_hdr r; size_t need; int ret; if (size - off < sizeof(r)) /* off <= size always holds here */ return -EINVAL; memcpy(&r, buf + off, sizeof(r)); off += sizeof(r); need = ((size_t)r.len + 3) & ~(size_t)3; /* cannot overflow: len <= 65535 */ if (size - off < need) return -EINVAL; ret = fn(r.type, buf + off, r.len, ctx); if (ret) return ret; off += need; } return off == size ? 0 : -EINVAL; /* reject trailing bytes */ }The parser keeps one invariant:
off <= size. Every check is written assize - off < need, which cannot overflow because of the invariant, whereasoff + need > sizecan wrap ifneedis huge and let a bad record through. The padded length is computed from a 16-bit value, so it fits easily insize_t. Headers are copied withmemcpy()into local structures, which avoids unaligned loads and strict aliasing problems on architectures that fault on them.The count is capped before the loop, so a header claiming four billion records cannot make the kernel loop for a long time, and the final check rejects trailing bytes. The callback's error is passed through so the caller can distinguish "bad format" from "valid record the device rejected".
The comment about the kernel copy is the point interviewers probe: the blob must be copied from user space once, with
memdup_user()and a size limit, and parsed only from that copy. Parsing directly from user memory would allow a second thread to change lengths after validation. Complexity is linear in the size of the blob. - read() and write() for a character deviceMedium · C
Write
open,readandwritefor a character device that stores up to 4096 bytes, like a small RAM file.readreturns data from*pposup to the bytes written so far and returns 0 at the end.writestores at*ppos, performs a short write if the request runs past the end, returns-ENOSPCwhen there is no space left, and extends the valid length. Protect the state with a mutex, handle signals while waiting for the lock and bad user pointers correctly, and recover the per-device structure from the inode. The solution uses the real kernel APIs; the test harness supplies small stand-ins for them.Hints
- Clamp count to what is available before copying, then advance *ppos only on success.
- copy_to_user() returns the number of bytes not copied, not an error code.
- Make sure every return path after taking the mutex releases it.
Solution and explanation
#define MYDEV_SIZE 4096 struct mydev { struct cdev cdev; struct mutex lock; /* protects data and len */ size_t len; /* bytes written so far */ char data[MYDEV_SIZE]; }; static int mydev_open(struct inode *inode, struct file *filp) { filp->private_data = container_of(inode->i_cdev, struct mydev, cdev); return 0; } static ssize_t mydev_read(struct file *filp, char __user *ubuf, size_t count, loff_t *ppos) { struct mydev *d = filp->private_data; ssize_t ret; if (mutex_lock_interruptible(&d->lock)) return -ERESTARTSYS; if (*ppos < 0) { ret = -EINVAL; } else if ((size_t)*ppos >= d->len) { ret = 0; /* end of file */ } else { count = min(count, d->len - (size_t)*ppos); if (copy_to_user(ubuf, d->data + *ppos, count)) { ret = -EFAULT; } else { *ppos += count; ret = count; } } mutex_unlock(&d->lock); return ret; } static ssize_t mydev_write(struct file *filp, const char __user *ubuf, size_t count, loff_t *ppos) { struct mydev *d = filp->private_data; ssize_t ret; if (count == 0) return 0; if (mutex_lock_interruptible(&d->lock)) return -ERESTARTSYS; if (*ppos < 0) { ret = -EINVAL; } else if (*ppos >= MYDEV_SIZE) { ret = -ENOSPC; } else { count = min(count, (size_t)(MYDEV_SIZE - *ppos)); /* short write */ if (copy_from_user(d->data + *ppos, ubuf, count)) { ret = -EFAULT; } else { *ppos += count; d->len = max(d->len, (size_t)*ppos); ret = count; } } mutex_unlock(&d->lock); return ret; } static const struct file_operations mydev_fops = { .owner = THIS_MODULE, .open = mydev_open, .read = mydev_read, .write = mydev_write, .llseek = default_llseek, };open()stores the device infilp->private_datausingcontainer_of()on the embeddedcdev, so one set of file operations serves many instances. Both I/O paths take the mutex withmutex_lock_interruptible()and return-ERESTARTSYSif a signal arrives, so a user can interrupt a process waiting on a busy device.read()returns 0 at the end of valid data, which is howcatknows to stop, and otherwise copies at most the remaining bytes.write()performs a short write at the end of the buffer instead of failing the whole request, which matches the POSIX contract that callers loop on partial writes, and returns-ENOSPConly when no byte fits. A failed copy returns-EFAULTand does not move*pposor change the length.The structure with a single exit releases the lock on every path, which the test checks after a fault. The copy functions can sleep, which is fine under a mutex but would be a bug under a spinlock, and a good candidate says so. Follow-ups usually ask how to add
poll()and blocking reads, and what happens if the device is unbound while the file is open, which needs the reference counting pattern from the driver model section. - Transmit descriptor ring with ownership bitsMedium · C
A network controller reads transmit descriptors from a ring in coherent DMA memory. Each descriptor has a 64-bit buffer address and a 32-bit control word whose top bit means "owned by the device" and whose low 16 bits hold the length. Write
tx_post(), which fills the next descriptor and hands it to the device, failing when the ring is full, andtx_clean(), which reclaims completed descriptors in order up to a budget and calls a completion function for each. The device clears the ownership bit when it has finished. Model the memory ordering the hardware needs with C11 atomics and explain the kernel barriers they stand for.Hints
- The device may read the descriptor the moment the ownership bit is set, so everything else must be written first.
- Stop cleaning at the first descriptor the device still owns, even if later ones look finished.
Solution and explanation
#include <stdatomic.h> #include <stdint.h> #define TX_RING 256u /* power of two */ #define DESC_OWN (1u << 31) /* 1: device owns the descriptor */ #define DESC_LEN 0xFFFFu struct tx_desc { /* shared with the device (coherent DMA) */ uint64_t addr; _Atomic uint32_t ctrl; /* OWN | length */ }; struct tx_ring { struct tx_desc desc[TX_RING]; void *cookie[TX_RING]; /* driver-only: buffer to free on completion */ uint32_t head; /* next slot the driver fills */ uint32_t tail; /* oldest slot not yet reclaimed */ }; /* Returns 0, or -1 if the ring is full (the caller stops the queue). */ int tx_post(struct tx_ring *r, uint64_t dma_addr, uint32_t len, void *cookie) { struct tx_desc *d; if (r->head - r->tail == TX_RING || len == 0 || len > DESC_LEN) return -1; d = &r->desc[r->head & (TX_RING - 1)]; d->addr = dma_addr; r->cookie[r->head & (TX_RING - 1)] = cookie; /* Like dma_wmb(): addr must be visible before the device sees OWN. */ atomic_store_explicit(&d->ctrl, DESC_OWN | len, memory_order_release); r->head++; /* A real driver now writes the tail/doorbell register with writel(). */ return 0; } /* Reclaim descriptors the device has finished with, oldest first. */ unsigned int tx_clean(struct tx_ring *r, void (*complete)(void *cookie, uint32_t len), unsigned int budget) { unsigned int done = 0; while (r->tail != r->head && done < budget) { unsigned int i = r->tail & (TX_RING - 1); /* Like dma_rmb() after reading OWN: no field read may move before it. */ uint32_t ctrl = atomic_load_explicit(&r->desc[i].ctrl, memory_order_acquire); if (ctrl & DESC_OWN) break; /* device still working on it: stop in order */ complete(r->cookie[i], ctrl & DESC_LEN); r->cookie[i] = 0; r->tail++; done++; } return done; }tx_post()writes the address and stores the cookie, then setsOWN | lenwith a release store. The release stands fordma_wmb()in a real driver: it guarantees the device cannot observe the ownership bit before the address. After posting, a driver writes the doorbell withwritel(), which already orders the coherent memory writes before the MMIO write, so no further barrier is needed there.tx_clean()loads the control word with acquire, standing for reading the ownership bit followed bydma_rmb(), so no field is read before the device has really released the descriptor. It stops at the first descriptor still owned by the device, because completions must be processed in ring order; skipping ahead would desynchronisetailfrom the hardware.Free-running
headandtailcounters give full capacity and correct wrap-around, which the test starts nearUINT32_MAXto prove. Interviewers ask next where the queue is stopped (when free slots fall below a worst-case packet), where it is woken (in clean-up once enough space returns), and how that stop and wake pair avoids a race in which both sides miss each other's update. - Build a scatter-gather list with device limitsHard · C
A user buffer starts
offsetbytes into its first page and islenbytes long. You are given the bus address of every page it touches (each page-aligned, 4096 bytes). The device accepts at mostmax_entssegments, each no longer thanmax_segbytes, and no segment may cross aboundaryline (a power of two such as 64 KiB, a common DMA engine restriction). Writebuild_sg()that fills an array of{addr, len}segments covering the buffer in order, merging physically contiguous pages when the limits allow, and returns the number of segments or -1 if more thanmax_entsare needed.Hints
- Handle the buffer one page-sized piece at a time; only the first and last pieces are partial.
- When a piece starts exactly where the previous segment ends, try to extend that segment first.
- The room left before a boundary is boundary minus the start address modulo boundary, minus the length already used.
Solution and explanation
#include <stdint.h> #define PAGE_SIZE 4096u struct sg_ent { uint64_t addr; uint32_t len; }; static uint64_t min3(uint64_t a, uint64_t b, uint64_t c) { uint64_t m = a < b ? a : b; return m < c ? m : c; } /* * Build a DMA segment list for a buffer that starts 'offset' bytes into the * first page and is 'len' bytes long. pages[i] is the bus address of the i-th * page (page aligned). Physically contiguous pages are merged, but no segment * is longer than max_seg or crosses a 'boundary' line (a power of two). * Returns the number of segments, or -1 if more than max_ents are needed. */ int build_sg(const uint64_t *pages, uint32_t offset, uint64_t len, uint32_t max_seg, uint64_t boundary, struct sg_ent *sg, int max_ents) { int n = 0; unsigned int pg = 0; while (len) { uint64_t addr = pages[pg++] + offset; uint64_t chunk = PAGE_SIZE - offset; if (chunk > len) chunk = len; len -= chunk; offset = 0; while (chunk) { struct sg_ent *last = n ? &sg[n - 1] : 0; uint64_t take; if (last && last->addr + last->len == addr) { /* extend the previous segment as far as its limits allow */ take = min3(chunk, max_seg - last->len, boundary - (last->addr & (boundary - 1)) - last->len); if (take) { last->len += (uint32_t)take; addr += take; chunk -= take; continue; } } if (n == max_ents) return -1; take = min3(chunk, max_seg, boundary - (addr & (boundary - 1))); sg[n].addr = addr; sg[n].len = (uint32_t)take; n++; addr += take; chunk -= take; } } return n; }The outer loop walks the buffer page by page: the first piece starts at
offset, later pieces start at 0, and the last is cut to the remaining length. For each piece, the inner loop either extends the previous segment, if the piece starts exactly where it ends, or opens a new one. Both are limited by the same two rules: the segment may grow tomax_seg, and it may grow only up to the next boundary line measured from its own start address. A piece that hits a limit part-way is split, and the rest continues in a new segment, so a 1 MiB contiguous buffer with a 64 KiB boundary produces 16 segments.Time is linear in the number of pages plus the number of segments, and memory is the output array only. The test runs twenty thousand random cases with random contiguity, offsets and limits, and checks byte by byte that the segments cover exactly the buffer and respect every limit.
In the kernel,
sg_alloc_table_from_pages()does the merging and the block layer applies the device'smax_segment_sizeandseg_boundary_masklimits. Interviewers ask this to see whether you can reason about boundaries and off-by-one errors, and whether you know those limits exist. - Sequence lock for a timestamped sampleHard · C
An interrupt path updates a 64-bit timestamp and a 64-bit value together, many thousands of times a second. Readers on other CPUs need a consistent pair, must never block the writer, and are allowed to retry. Implement a sequence lock:
seq_write()andseq_read(), with writers serialised by a mutex in this user-space version. Make it correct under the C11 memory model, so the test passes under ThreadSanitizer, and explain how it maps to the kernel'sseqlock_tandseqcount_t.Hints
- The counter is odd while a write is in progress; a reader retries if it saw an odd value or the value changed.
- In C11 the data fields must be atomics accessed with relaxed ordering, or the racing reads are undefined behaviour.
- The reader needs an acquire fence between its data loads and the second counter load.
Solution and explanation
#include <pthread.h> #include <stdatomic.h> #include <stdint.h> /* Readers never block the writer; they retry if a write overlapped their read. */ struct seq_sample { _Atomic uint32_t seq; /* odd while a write is in progress */ _Atomic uint64_t ts; /* data fields are atomics so the racy */ _Atomic uint64_t value; /* reads are defined behaviour in C11 */ pthread_mutex_t wlock; /* serialises writers */ }; void seq_write(struct seq_sample *s, uint64_t ts, uint64_t value) { pthread_mutex_lock(&s->wlock); uint32_t q = atomic_load_explicit(&s->seq, memory_order_relaxed); atomic_store_explicit(&s->seq, q + 1, memory_order_relaxed); atomic_thread_fence(memory_order_release); /* seq odd before data changes */ atomic_store_explicit(&s->ts, ts, memory_order_relaxed); atomic_store_explicit(&s->value, value, memory_order_relaxed); atomic_store_explicit(&s->seq, q + 2, memory_order_release); /* data before even */ pthread_mutex_unlock(&s->wlock); } void seq_read(struct seq_sample *s, uint64_t *ts, uint64_t *value) { uint32_t q1, q2; do { q1 = atomic_load_explicit(&s->seq, memory_order_acquire); *ts = atomic_load_explicit(&s->ts, memory_order_relaxed); *value = atomic_load_explicit(&s->value, memory_order_relaxed); atomic_thread_fence(memory_order_acquire); /* data reads before re-check */ q2 = atomic_load_explicit(&s->seq, memory_order_relaxed); } while ((q1 & 1) || q1 != q2); }The writer makes the counter odd, writes the data, then makes it even again. A reader samples the counter, reads the data, samples again, and accepts the result only if both samples are equal and even, which proves no write overlapped the read.
The ordering is the subtle part. On the write side, a release fence after making the counter odd stops the data stores from becoming visible before the odd value, and the final release store stops them leaking past the even value. On the read side, the first counter load is acquire, so data loads cannot move before it, and an acquire fence before the second load stops the data loads from moving after it. The data fields are relaxed atomics, because in C11 a plain racing read is undefined behaviour even if the result is thrown away. The test checks that
value == ts * 3 + 7for every pair read by three readers during three million writes.In the kernel,
write_seqlock()combines the counter with a spinlock,read_seqbegin()andread_seqretry()contain the barriers, and the timekeeping code is the classic user. Readers must not follow pointers they read, because the object may be freed; that case needs RCU instead.
Design and debugging scenarios
Talk through each one out loud for 15–20 minutes, then compare your answer with the model walk-through.
- Design a Linux driver for a PCIe data acquisition cardSystem design
A PCIe card digitises analogue signals and streams samples at a sustained 2 GB/s. It has a DMA engine that writes into host memory using a ring of descriptors, MSI-X interrupts, and a set of control registers in BAR0. A user-space application must receive every sample without gaps and must be able to tell when data was lost. Design the driver: interfaces, memory, interrupts, and how you would test it.
Model walk-through
Requirements first. I would ask about the sample rate per channel, acceptable latency, whether there is one consumer or several, whether loss is ever tolerable, and what the IOMMU situation is on the target servers. I assume one consumer process, no gaps tolerated, and that latency of a few milliseconds is fine.
Data path. Copying 2 GB/s through
read()would cost a full CPU core incopy_to_user(), so I would avoid copies. The driver allocates a large ring of DMA buffers, for example 64 buffers of 4 MiB, usingdma_alloc_coherent()or streaming mappings with explicit sync calls if the platform is not cache coherent. User space maps the whole ring once withmmap(), implemented withdma_mmap_coherent(). The device fills buffer after buffer and writes a completion record, or the driver reads a hardware write pointer.Control interface. A char device with a small ioctl set: configure (sample rate, channels), start, stop, and "wait for buffers", which blocks until at least one buffer is full and returns the producer index and a sequence number.
poll()support lets the application use epoll. The application returns buffers by advancing a consumer index through an ioctl or a shared page.Interrupts. One MSI-X vector for "buffer complete", with interrupt moderation in the device so we get one interrupt per buffer, not per descriptor. The handler reads the hardware pointer, updates the producer index, and wakes the wait queue. No threaded handler is needed because nothing sleeps.
Overflow and loss. If the consumer falls behind and the device reaches a buffer the application has not returned, the device must stop or drop. I would have the device skip and count the dropped data, and expose the counter in each buffer's header and in sysfs, so the application can detect gaps through sequence numbers instead of silently getting overwritten data.
Probe and lifetime. Standard PCI probe order: enable, request regions, map BAR0, set bus mastering, 64-bit DMA mask, allocate vectors, allocate the ring, and create the char device last. On remove or surprise removal, stop the DMA engine, wake all waiters with an error, and keep the ring memory alive until the last mapping and file reference are gone, using a kref.
Failure modes and testing. Test sustained throughput with the application pinned to a CPU near the device's NUMA node, induce consumer stalls to check loss accounting, unplug or reset the card while streaming, and run with KASAN, lockdep and DMA API debugging. A hardware loopback or test pattern mode, such as a counter written as samples, makes gaps and corruption trivial to detect.
- Intermittent receive data corruption on an ARM boardDebugging
Your SPI controller driver uses DMA for transfers over 64 bytes. On an x86 test system with a PCIe SPI adapter it is perfect. On the production ARM SoC, roughly one transfer in ten thousand under heavy load returns data with a 64-byte block of stale bytes, always aligned to 64 bytes. Smaller transfers are always correct. How do you find and fix the cause?
Model walk-through
Read the symptoms. Corruption in 64-byte aligned blocks matches the cache line size of the ARM cores. Only DMA transfers fail, only on the platform without hardware coherency, and only under load. Everything points to cache maintenance: the CPU is reading a cache line that holds stale data instead of what the DMA engine wrote to memory. x86 hides this because its DMA is coherent.
Hypotheses, in order of likelihood.
- A missing or misordered sync. The driver reads the receive buffer before
dma_unmap_single()ordma_sync_single_for_cpu(), or calls them with the wrong direction or length. - Cache line sharing. The receive buffer is embedded in a structure, or allocated next to data the CPU writes during the transfer. When that neighbouring data is written back, it overwrites the line the device filled, or the invalidate discards the CPU's write.
- Speculative prefetch. The CPU touches the buffer, or a neighbour, while the device owns it, so a line is brought back into the cache after the invalidate at map time and before the device finished. Only
for_cpusyncs after the DMA completes prevent this. - The completion is signalled before the data is in memory, for example the driver trusts a status bit read with a relaxed accessor and reads the buffer before the DMA write has landed.
How to prove it. Enable
CONFIG_DMA_API_DEBUG, which reports CPU access to device-owned buffers and mismatched unmap parameters. Review every path from map to CPU read. Print the buffer's address and alignment: if it is not cache line aligned, or the length is not a multiple of the line size, hypothesis 2 becomes likely. A test that fills the buffer with a known pattern before mapping shows whether the stale data is old CPU data (sharing or prefetch) or simply the pattern (missing invalidate).Fix. Allocate receive buffers separately with
kmalloc(), which guarantees DMA-safe alignment, or use a dedicated DMA-safe bounce buffer for small transfers; follow the ownership rules exactly, withdma_unmaporsync_for_cpuafter the completion and before any CPU access; and usereadl()rather than a relaxed read for the completion status so later buffer reads are ordered after it. Many SPI core helpers already manage mapping; using them instead of hand-rolled mapping removes whole classes of these bugs.Verify. Run the load test for many times the previous failure interval, on several boards, with DMA API debugging on and then off, and add the pattern test to the regression suite.
- A missing or misordered sync. The driver reads the receive buffer before
- System hangs on resume after an I2C sensor driver was addedDebugging
After adding a driver for an I2C accelerometer with a data-ready interrupt, the board hangs on roughly one resume in fifty. With a serial console you see the start of the resume messages, then nothing; sometimes, instead, you get "BUG: scheduling while atomic" with the sensor's interrupt handler in the trace. Walk through your investigation.
Model walk-through
Two symptoms, probably one cause. "Scheduling while atomic" from the sensor's interrupt handler means the handler slept in atomic context. On an I2C device the obvious candidate is a register read over I2C from the hard interrupt handler. Most of the time it may happen to work if the I2C controller driver happens not to sleep, but during resume the controller is often still suspended, so the transfer waits for it to resume and sleeps, which explains both the warning and the hang.
Confirm. Read the trace: if
i2c_transferorregmap_readappears under the handler, that is the bug. Check how the interrupt was requested:request_irq()with a handler that talks to the chip, instead ofrequest_threaded_irq()withIRQF_ONESHOT. EnableCONFIG_DEBUG_ATOMIC_SLEEPand lockdep, which should turn the intermittent failure into a reliable warning on the first interrupt.Second cause to check: interrupt during the resume window. Even with a threaded handler, the sensor may assert data-ready while the system is suspending or resuming. If the handler runs before the I2C controller has resumed, the transfer fails or waits. Look at the order in the trace with ftrace events for the PM core and the I2C bus, using
pm_print_timesorinitcall_debugto timestamp each device's callbacks.Fix.
- Use
devm_request_threaded_irq()with a NULL primary handler andIRQF_ONESHOT, and do all I2C work in the thread. - In suspend, tell the sensor to stop generating data-ready and call
disable_irq(), which waits for a running handler to finish. In resume, restore configuration, clear any pending status, thenenable_irq(). - If the sensor is a wake source, use
enable_irq_wake()and handle the wake event after the bus has resumed. - Make sure the device link between the sensor and the I2C controller exists, which fw_devlink usually provides, so the sensor suspends before and resumes after its bus.
Prove the fix. Run several thousand automated suspend and resume cycles, for example with
rtcwakein a loop, with the sensor streaming data at its highest rate and debug options enabled. Keep the stress test in the regression suite, because these bugs come back when someone refactors the interrupt path. - Use
- Review a patch for a platform driverCode review
A colleague asks you to review this extract from a new platform driver for a small FIFO peripheral. The device raises its interrupt when data arrives. Find the problems and say how to fix each one.
static irqreturn_t fifo_irq(int irq, void *data) { struct fifo_dev *fd = data; u32 v = readl(fd->base + FIFO_DATA); mutex_lock(&fd->lock); fd->buf[fd->head++] = v; mutex_unlock(&fd->lock); wake_up(&fd->wq); return IRQ_HANDLED; } static int fifo_probe(struct platform_device *pdev) { struct fifo_dev *fd = kzalloc(sizeof(struct fifo_dev), GFP_KERNEL); request_irq(platform_get_irq(pdev, 0), fifo_irq, IRQF_SHARED, "fifo", fd); fd->base = devm_platform_ioremap_resource(pdev, 0);Model walk-through
I would group the findings by severity, starting with those that crash or corrupt memory.
Sleeping in a hard interrupt handler.
mutex_lock()can sleep, and the handler runs in atomic context. With debug options this warns at once; without them it can deadlock. The lock protecting the buffer must be a spinlock:spin_lock()in the handler andspin_lock_irqsave()in the read path. Alternatively, if the reader is a single consumer, a lock-free ring with acquire and release ordering avoids the lock entirely.Buffer overflow.
fd->head++is never bounded or wrapped. Onceheadpasses the buffer size, the handler writes past the end of the structure. Use a ring with a mask orkfifo, and decide what to do when it is full: drop the sample and count it, or stop accepting data.The interrupt is requested before the registers are mapped. With
IRQF_SHARED, the handler can be called as soon asrequest_irq()returns, even for another device's interrupt, and it would read throughfd->base, which is still NULL. Map registers, initialise the lock, wait queue and buffer, and quiesce the device first; request the interrupt after that.No error checking.
kzalloc()can fail,platform_get_irq()can return a negative error,devm_platform_ioremap_resource()returns anERR_PTR, and the return value ofrequest_irq()is ignored. Each must be checked, with deferral handled throughdev_err_probe().Shared interrupt handling is wrong. A handler on a shared line must check its device's status register and return
IRQ_NONEif its device did not interrupt. This one always reads the data register, which may also pop the FIFO at the wrong time, and always claims the interrupt.Lifetime and leaks.
kzalloc()andrequest_irq()are unmanaged while the mapping is managed, so error paths leak and remove order is muddled. Usedevm_kzalloc()anddevm_request_irq()consistently, after making sure that remove disables the device's interrupt first and that no file can still reference the memory, or use reference counting if a char device is exposed.Smaller points. Use
sizeof(*fd), store driver data withplatform_set_drvdata(), and consider whether one register read per interrupt is enough: if the FIFO can hold several words, the handler should drain it in a loop until a status bit says it is empty, or data is lost when interrupts are coalesced.
Run a mock interview
Ask a friend to play the interviewer, or record yourself. Pick questions and an exercise you have not opened yet.
| Minutes | Stage | What to do |
|---|---|---|
| 0–5 | Introduction and background | Give a two-minute summary of one driver you wrote or changed: the device, the bus, the hardest bug and how you proved the fix. The interviewer notes which areas to probe. |
| 5–20 | Kernel knowledge round | The interviewer picks one question from each of the interrupts, locking and DMA sections and keeps asking why. Answer by naming the execution context first, then the rule, then the failure if the rule is broken. |
| 20–40 | Coding in C | Solve one Medium exercise you have not opened, in a plain editor with no autocompletion. Talk through invariants, ordering and edge cases as you write, then compile with warnings and run a small test you write yourself. |
| 40–50 | Debugging or code review | Work through one scenario out loud: list hypotheses in order of likelihood, say which tool or experiment separates them, then propose the fix and how you would prove it. |
| 50–55 | Design follow-up | The interviewer changes one requirement in the scenario, such as adding a second consumer or removing the IOMMU. Explain what changes in the design and why. |
| 55–60 | Your questions and review | Ask about the hardware, the upstream policy and how the team tests on real boards. Afterwards, write down every question where you hesitated and revisit that section within two days. |
Cheat sheet
Facts worth having at your fingertips on the day.
- Hard IRQ handler
- Atomic context: no sleeping, no mutexes, no
GFP_KERNEL, nocopy_to_user(); returnIRQ_HANDLEDorIRQ_NONE. - Threaded IRQ with NULL primary
- Requires
IRQF_ONESHOTso the line stays masked until the thread returns. - spin_lock_irqsave()
- Use in non-interrupt code for any lock that a hard interrupt handler also takes.
- spin_lock_bh()
- Use in process context for a lock shared with softirqs, tasklets, timers or NAPI poll.
- GFP_KERNEL vs GFP_ATOMIC
- GFP_KERNEL may sleep and reclaim; GFP_ATOMIC never sleeps and can fail, so handle NULL.
- copy_to_user()
- Returns the number of bytes not copied; turn non-zero into
-EFAULT. May sleep. - devm_ resources
- Released in reverse order after
remove()returns or when probe fails. - readl() / writel()
- One access of the right width, little-endian, with ordering against normal memory; relaxed variants drop that ordering.
- Coherent DMA
dma_alloc_coherent(): shared at any time, uncached on non-coherent systems; use for rings and queues.- Streaming DMA
dma_map_single()with a direction; the device owns the buffer until unmap orsync_for_cpu; checkdma_mapping_error().- Descriptor ordering
- Write fields,
dma_wmb(), set the ownership bit; read the ownership bit,dma_rmb(), read the fields. - DMA mask
dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)), falling back to 32 bits; otherwise swiotlb bounces.- All-ones read on PCIe
- 0xFFFFFFFF usually means the device is gone or not responding; stop and fail requests.
- MSI-X
- One vector per queue, not shared, with CPU affinity; allocate with
pci_alloc_irq_vectors(). - NAPI
- Interrupt schedules poll, poll processes up to the budget,
napi_complete_done()then re-enable interrupts. - Runtime PM
pm_runtime_resume_and_get()before access,pm_runtime_put_autosuspend()after; balance every get.- Debug options
- KASAN, lockdep, DEBUG_ATOMIC_SLEEP, KCSAN, UBSAN, kmemleak and DMA_API_DEBUG during development.
- Windows IRQL
- DISPATCH_LEVEL behaves like atomic context: no waiting and no pageable memory.
Red flags that sink candidates
- Saying "I would just add a mutex" without asking which contexts take the lock. Always name the contexts first; if an interrupt handler is one of them, the answer is a spinlock with interrupts disabled on the other side.
- Treating
volatileas a synchronisation tool. It stops some compiler optimisations, nothing more. Explain the barrier or atomic operation that gives the ordering you need, and usereadl()andwritel()for registers. - Answering DMA questions as if all systems were cache coherent like x86. Interviewers at SoC companies expect you to talk about cache maintenance, ownership rules and cache line alignment unprompted.
- Ignoring error paths and lifetimes. A probe without unwinding, a remove that frees memory before stopping the interrupt, or file operations that outlive devm_ memory are the bugs reviewers look for first. Walk through teardown as carefully as setup.
- Waiting on hardware without a timeout. Every poll loop and completion wait needs a bound and an error path, because devices hang, get unplugged and return all ones.
- Inventing a custom interface where a subsystem exists, such as a private char device for a sensor that belongs in IIO or a network device driven through ioctl. Say which subsystem you would use and why.
- Debugging by adding printk until the bug disappears. Describe a method: reproduce, form hypotheses, use the tool that separates them (lockdep, KASAN, ftrace, DMA API debugging), then prove the fix under stress.
FAQ
How do I prepare for a Linux device driver interview?
Work on three things in parallel. First, C fluency without a standard library: pointers, bit manipulation, intrusive lists, ring buffers and concurrency with atomics, practised under time pressure. Second, kernel concepts explained out loud: execution contexts, locking choices, DMA ownership, the driver model and power management, which this page's sections cover. Third, real code: write a small driver for a device you can test, such as a QEMU emulated device, an I2C sensor on a development board, or a virtual char device, and read one mature driver in the kernel tree for the device class the job involves.
Before the interview, prepare two or three stories about bugs you found and fixed, with the tools you used and how you proved the fix. Those stories carry the behavioural part of most driver interviews.
What coding questions are asked in device driver interviews?
Mostly C problems close to driver work rather than abstract puzzles: bit and register manipulation, endianness and parsing of binary formats, linked lists with container_of, ring buffers, bitmap allocators, memory allocators, and string or buffer handling with careful bounds. Senior loops often add a concurrency problem, such as a lock-free queue or a reader-writer scheme, and ask about memory ordering.
Some companies also ask you to write or sketch kernel code on a whiteboard: a char driver's read and write, an interrupt handler with deferred work, or a probe function with error handling. You will not be marked down for an imperfect API name if the structure, context rules and error handling are right. General algorithm questions appear too, usually at an easier level than in pure software roles.
Do I need upstream kernel contributions to get a driver job?
No, but they help a great deal. Many driver engineers work on product kernels and never send patches upstream, and interviewers know that. What they look for is evidence that you have worked with real kernel code: debugging an oops, bringing up hardware, changing a driver and understanding the consequences.
Upstream patches are strong evidence because they have passed public review, and they give you concrete material for interviews. Even a few small, accepted fixes, such as error path corrections or conversions to newer APIs in a driver you can test, show that you can read kernel code, follow the process and take review. If you do not have them, a well-documented personal driver project with a clear README and tests serves a similar purpose.
How is a device driver role different from a firmware or embedded systems role?
A driver developer usually works inside an operating system kernel, mostly Linux, and the job is to make hardware work through that kernel's frameworks: the driver model, DMA API, interrupt subsystem, power management and the device class subsystems. Interviews therefore test kernel concepts and concurrency heavily.
A firmware developer works on bare metal or a small RTOS, often owning the whole software image of a microcontroller, and interviews focus on startup code, peripherals, memory limits and real-time behaviour. An embedded systems engineer role is broader and may include hardware, firmware, Linux user space and system integration. There is plenty of overlap, especially in C, hardware reading and debugging. See the Firmware Developer and Embedded Systems Engineer pages for those interviews.