Developing software in a device design company - full version
Most people associate the brand Haltian with high quality device hardware development. It is perhaps not as well known that we also have a reasonably large software team. I have been part of the team already more than a year so I have some insights what it is like to be a software professional in a device design company.
Many of you think of programming as the core of software development. In need to agree to some extent: while there are many other activities during the day, it is programming that usually sets us apart from the rest.
A newcomer may be interested in the selection of programming languages that are needed in order to qualify developing software for various devices. Well, it depends. In the Github, a popular open source repository, the 27 most popular programming languages have more than 5000 active projects each. Javascript and Python are getting traction over PHP and Ruby in the web apps development. Java is still used very widely in enterprise software and Android devices. C# lives in the Windows PC and Azure cloud.
While there are always some amount of cloud and PC software under work, the bread and butter for us is the device firmware and applications. Device software is still ruled by the plain old C++ and C languages. Of course there are many scripting and data languages applied as needed: shell scripting, Json and SQL just to name a few. User interfaces are sometimes written in QML which is a flavor of Javascript. It is usually not enough to pull out those lines of code from the sleeve. A lot of the daily routine is the usage of various tools for professional software projects. Some project might use Bitbucket as a code repository, Git as version control system, Gerrit for code review, Jira for bug reporting, and Jenkins for automatic build verification. Another set of tools could be used to check the inner characteristics of software. Valgrind is used to analyze the memory consumption, Perf for checking the processing speed, GDB and Lauterbach for understanding the code behavior and locating bugs. These are just examples - there are dozens of alternatives. An important tool is written by the developers themselves: the test code.
It is quite different to develop code a simple microprocessor than a high-end computer chip. For the latter one, it is a common practice mix-and-match a lot of existing open source software with your own applications instead of developing everything from scratch. If I could name one thing that has changed the way software is developed in the last couple of decades I would say it is open source and the way it is distributed. Naturally, this change has also sneaked into the device development.
From software perspective, a typical consumer device is not that different from, say, Raspberry Pi and Arduino. Let's compare these two popular kits. They are both available for everyday tinkerers in bargain prices but there are fundamental differences. Arduino is relatively simple microcontroller environment. You can use for example to develop your own electronic weather station or backyard lighting controller. How about a high fidelity media station? Not so much. Raspberry Pi, on the other hand, is a full blown computer board. I have seen Raspberry projects ranging from trail camera to a full blown smartphone. The default operating system of the Pi is Raspbian - a full blown Linux distribution.
The same applies to typical device development projects. If the device has only very simple microcontrollers, then we might develop most of the software ourselves. Just like the Arduino tinkerers. For a more full blown processor we might opt for a suitable Linux distribution and get the benefits of the open source ecosystem. For a mid-size system such as our Thingsee there are medium size operating systems available as well. In each of these cases there is usually a fair amount software to be created ourselves.
Let me elaborate on the open source. What does it mean for the software developers? First of all, open source is free. It is free just as the Free Beer: you didn't have to pay but you may get addicted :-) A very important point is the license, though. You have to obey the rules of the license, otherwise you may get into trouble. Another key point to know about open source is that it is valuable. James Bromberger has counted the amount of work put to Debian Wheezy distribution. This is a modest 19 billion dollars (19.000.000.000$) with typical developer salaries. The distribution runs code in 31 languages all the way from historical ones such as Cobol and Modula3. Let's pick up a popular SQL library called MySQL. That single library is worth 64 million. All this and much more with a bargain price of zero dollars.
The evolution of the open source has certainly changed the daily routine of us software developers over the years. We can stand on the shoulders of a giant to create big systems in acceptable time. We can play a detective to find the best components. We can learn from the great masters by studying their code. We can mix and match existing libraries and applications with our own software to create our own distributions.
Creating a distribution (or an image) for special purpose devices using open source is a form of art in itself. There is a full pipeline of tools with the sole purpose to collect those damned lines of code from repositories around the world and merge them together into one consistent software image. That image can then be installed to the device in order for it to function.
Distributions... It all started years ago when it was found that sometimes the compiler is not enough. Smart people created “makefiles” that were used to simplify generating a functional software from a big project of source code and related files such as data, texts, and images. After a while it was found that makefiles tend to get too complex, so a fleet of automatic tools to generate the makefiles emerged. Tools such as CMake, QMake, and Autoconf take simpler configuration descriptions and generates the makefiles for you. If you are developing a single application or a library this may be enough. Creating a full distro requires at least one more layer of tools.
When creating software for devices instead of a regular computer, it is typical to create a very specific distribution. Two popular distribution creation tools for embedded systems are Buildroot and Yocto. Yocto, for example, uses Bitbake to interprete special "baking" recipes to cross-compile a selection of software from various repositories for your chosen hardware environment. It includes tooling to generate flash-ready system images and installable application packages. Sounds simple. Simple it is, if the selected software was designed to run in similar environment as yours. And designed for cross-compilation. This is not always the case. Luckily both Buildroot and Yocto have some beautiful recipes ready-made. If not, you may have to develop a recipe yourself. And usually not only the recipe. You may have to create some patches, too.
Patch is a change. It is a special kind of change that you can store separately to the rest of the software. In the best case you could contribute the patch as part of the original code repository. You could also store the patch with your recipe, but you should continue to respect the license of the underlying software. You could patch code lines, configuration files, maybe even the data files. In the process, you'd have the opportunity to dive deep into the internals of the software you are working with. Don't be surprised, if you meet some Fortran code one day, a language invented 62 years ago. What if you end up spending a full week patching a single library? It would still be a small effort compared to the time spent by the original developers who contributed the code in the first place. Why not join them by contributing your change, whether or not mandatory?
There can be huge difference in the software stack in various kinds of devices, but the software is always somehow connected to the rest of the hardware such as USB port, WiFi, SD card, internal memory, internal clocks, and the like. A lot of teamwork is happening with our friends at the hardware department who design the circuit boards, component assembly, mechanics, antennas, manufacturing necessities and so forth. Look no further than the following features: firmware update, device startup procedure, control of the hardware devices. These software features can be quite specific to each hardware.
Software developers tend to specialize, even in the device development. Some of us are great in user interface programming while others reach their heights when getting the device drivers alive and kicking. There is expertise in image processing libraries and internet connectivity. Artificial intelligence and machine vision. I have colleagues who like to put the system into the toughest of tests to find every possible weakness, while I myself tend to specialize on the early steps of the project where the focus is on fast trialing of new ideas.
Full of tech apparently. How about the soft skills? Personally, I have always thought programming is just one of my interests. I like to read and write. Visual arts is my lifetime devotion. In fact, there are so many areas of life I would like to participate. To be honest it was quite a concern to me when I started as a software dude long time ago. How to keep up with any of these "secondary" interests?
Luckily, Haltian is full of similar minded people. We have excellent artists and men of science. Some of us are great in front of big audience. And there are also possibilities to apply some of these skills. Just think about a beautiful user interface design or great industrial design. What a piece of art! Is there some cool mathematics software created inside a modern scientific instrument? For sure. What about the much hyped IoT devices or some cool wearables? Need some more life sciences talents?
I can't deny that software development is sometimes hard work. That applies to Haltian as well. On the other hand there exist a sincere interest to balance the workload. Tired people create crappy software. In order to stay creative, you need to pursue those other interest. In my experience, working at Haltian all this is possible.
Cheers,
Jyrki