Android smartphones and tablets represent only the most visible part of a larger embedded Linux ecosystem that extends throughout industries like consumer electronics, telecom, manufacturing, and transportation. As device requirements call for more intelligence and networking savvy, Linux has become the oft-hidden secret sauce in everything from set-top boxes to home automation systems to rugged field handhelds.
It’s not surprising then that embedded Linux developers continue to be in high demand. But just who are these people, what skills do they need, and how do they spend their days? Are they primarily programmers writing to hardware targets or are they mechanical and electrical engineers who know some coding?
The good news for developers who want to check out embedded is that software is increasingly running the show. With multi-faceted system-on-chips (SoCs) running today’s gadgets, “most of the deep EE issues have been abstracted away in digital circuits,” says Justin Waters, Manager, Embedded Development at Timesys, which offers its LinuxLink development platform. “The remaining issues are much more software-oriented.”
Still, most embedded applications require someone with deep understanding of hardware design. In addition, many projects demand knowledge of hardware tools like soldering irons, oscilloscopes, target debuggers, and logic and network analyzers.
“It’s often said that software dominates the embedded world these days, but that varies considerably according to the project and industry,” says Rick Lehrbaum, a consultant and publisher of the DeviceGuru blog. Lehrbaum co-founded AmPro, now part of Adlink, and is known as the chief developer of the PC/104 embedded board form-factor.
The Job Description
“Embedded development” encompasses a wide range of potential jobs, from hardware design to firmware development to COTS-based system development. Arguably, Android app programming could be included, although it often has more in common with general software development.
Developing firmware requires especially demanding skills, says Lehrbaum. “Firmware developers need to understand lower-level hardware function control,” he says. “At the same time, they need to know issues like user interface design and system reliability.”
Firmware developers, who often work hand in hand with hardware engineers, need a thorough knowledge of the Linux kernel and components, and possibly Android expertise as well. They also need to know at least one processor architecture. In consumer electronics, this is typically ARM, but in other areas, especially networking and industrial, x86 is far more prevalent, along with MIPS or PowerPC. Increasingly, developers are asked to optimize code for multi-core processors, which in turn demands a deeper understanding of multitasking.
Most embedded development jobs require an understanding of hardware design, prototyping, compiling, debugging and testing tools. It is also important to be able to write interfaces to multiple peripherals, including serial busses, timers, generators, DACs, and display devices. Your dreams will be filled with device drivers.
Projects based on off-the-shelf COTS hardware, using embedded module form-factors such as PC/104, COM Express, and QSeven, generally require less time and expertise than custom hardware projects. Using larger carrier-board formats with real-world ports like Mini-ITX reduces the learning curve even further.
“For COTS developers, the effort consists mostly of software development,” says Lehrbaum. “But there’s still the need for some knowledge of mechanical engineering, such as packaging, as well as electronic design, including custom I/O interface development.”
COTS developers are facing new challenges as modules move from x86 to ARM. “The transition to ARM is a big challenge and opportunity,” says Lehrbaum. “Most ARM chips are highly customized, whereas x86 is more of a well-defined system.”
Team Players
Embedded projects tend to be more diverse in scope than server- or desktop-oriented projects. In consumer electronics in particular, there is a continuing churn of new platforms, applications, components, processors, peripherals, and features, any of which can trip up the others.
“The biggest challenge is the sheer size of the market,” says Waters. “There are so many different architectures, operating systems, and component vendors, that it’s impossible for any one person to master it all. The embedded world is so heterogeneous that it’s very difficult for somebody to find a niche.”
Due to the variety of hats required, embedded development is usually performed in teams, with different members playing specialized roles, says Lehrbaum. In addition, different industries and device targets often demand specific knowledge.
“Embedded development is all about the target device,” says Lehrbaum. “Environmental factors and standards vary greatly depending on the industry.” For example, one needs to know about SAE for automotive applications, MIL-STD for military systems, and HIPAA for medical devices, he adds.
Despite the need for specialization, all the developers need at least a basic understanding of all development facets. Even mobile app developers using a higher-level Android framework, for example, need a grounding in power management, sensors, telephony, storage, and security. For mobile devices, power management consumes a lot of development time.
Troubleshooting: Diggers Wanted
The wide variety of components, skills, and tasks required in embedded development can be maddening, but this can also make it more compelling than standard software development. In addition, many developers like the sense of fulfillment they get from a finished device.
Your satisfaction may vary, however, if you become easily bored or frustrated with debugging and testing, which always seems to take longer with embedded. Many developers relish the forensic challenge of hunting down and squashing obscure bugs while many others simply endure it as a necessary evil.
“Embedded troubleshooting often means interacting with the system at the chip-, interface-, and protocol levels,” says Lehrbaum. “Developers need to understand the hardware enough to build fail-safe code and control the system so it doesn’t violate parameters.”
According to Waters, embedded debugging is the largest missing skillset among entry-level developers. “Most college coursework focuses on software development using a full-fledged debugger, but with all of the moving parts of a low-level system, these tools aren’t terribly useful,” he says. “A developer needs to truly understand how the pieces interact and come up with specialized ways to find the source of an issue. It’s important to be able to ‘think around’ difficult problems and function outside of a traditional debugging environment.”
This heuristic, trial-and-error process tends to self-select for a particular embedded hacker temperament. “The most successful embedded developers I know are what I call ‘diggers’,” says Waters. “They have the patience to dig in to a difficult problem and get down to the basics of what could be going wrong; they have the desire to peel away the abstractions. By contrast, for a higher-level programmer, this can be a distraction — it’s often helpful for them to ignore the implementation.”
Open Source Challenges and Benefits
Due to the crazy-quilt nature of open source and distributed resources, embedded Linux development tends to be more complex compared to a proprietary OS. Yet, open source more than pays for that complexity in other ways, say Waters and Lehrbaum.
“The wealth of sample open source code for all types of devices is invaluable,” says Waters. “It allows developers to leverage existing code with minimal changes. There’s also a large focus on portability and standardized interfaces.”
The open source community is another great asset. “Chances are, you aren’t the only one having the same problem,” says Waters. “Unless you’re on the bleeding edge, there’s a good chance the issue has already been fixed.”
To get more information on what it takes to be a Linux developer, check out the Linux Foundation’s Linux training site.
5 signs you might like embedded development:
