Google’s recent announcement that it had ported its open source TensorFlow machine intelligence (ML) library for neural networking to the Raspberry Pi was the latest in a series of chess moves from Google and its chief AI rival Nvidia to win the hearts and keyboards of embedded Linux developers. The competition is a part of a wider battle with Amazon, Microsoft, Intel, and others to bring cloud analytics to the edge in IoT networks to reduce latency, increase reliability, and improve security.
Thanks to a collaboration with the Raspberry Pi Foundation, the latest TensorFlow 1.9 release can now be installed on Raspberry Pi 2 or 3 SBCs from pre-built binaries using Python’s pip package system. Raspbian 9 users can install it with two simple commands.
Integrating TensorFlow models into an embedded project offers further challenges. Yet, as Google has shown with its AIY Projects kits for the Raspberry Pi, you can add a bit of ML smarts in Raspberry Pi based robots, vision systems, and other embedded gear without a huge learning curve.
The TensorFlow port should be particularly welcome in the Raspberry Pi educational community. As the RPi Foundation’s Eben Upton wrote in a congratulatory tweet about the “massive news,” the TensorFlow port will enable “cool machine-learning educational content.”
TensorFlow was essentially born to run on Linux, but on servers or desktops, not on a modest SBC like the Raspberry Pi. It now runs on all major server and desktop platforms and has been ported to Android and iOS. Yet, the Raspberry Pi was a particularly gnarly challenge, writes Google TensorFlow developer Pete Warden in the announcement. It wasn’t even possible until the Raspberry Pi 2 and 3 came along with faster quad-core processors.
A year ago, Warden and his team managed to cross-compile TensorFlow on the RPi 3, but it was a slow, complicated, crash-prone process. The new ability to install from pre-built binaries now makes it feasible for a much wider group of developers to join the party.
While Google’s AIY Projects was attempting to squeeze a cloud-based platform onto a simple hacker board, its team started with low-cost cardboard constructed kits with add-on boards for connecting with Google Cloud related embedded technologies. These include the AIY Vision Kit for the Raspberry Pi Zero W and WH, which performs TensorFlow-based vision recognition. It incorporates a “VisionBonnet” board with an Intel Myriad 2 neural network accelerator chip. AIY Projects also launched an AIY Voice Kit with the same RPi Zero WH target that lets you build a voice-controlled speaker with Google Assistant support.
As noted in this Hackster.io post about the port from Alasdair Allan, the AIY Vision Kit has struggled to perform well when operating locally. The Voice Kit has done better due to its greater reliance on Google Cloud.
Google’s Edge TPU accelerator
According to a Warden tweet following the announcement, TensorFlow is not currently tapping the potential ML powers of Broadcom’s VideoCore graphical processing unit, as Nvidia does with its more powerful Pascal GPU. He goes on to suggest that there might be potential for developing a special GPU-related port for the single-core Raspberry Pi Zero boardlets, but for now there’s sufficient power on the Pi’s four CPU cores. Speaking of potential hooks to the GPU, he writes: “With quad-core CPUs and Neon on the latest Pi’s, there’s not as big an advantage, though it’s still interesting on Pi Zeroes.”
Another interpretation is that Google is skipping the GPU because it expects Raspberry Pi users and other embedded developers to tap its recently announced, Linux-friendly Edge TPU ML accelerator chip for TensorFlow. The Edge TPU will be offered this fall along with an NXP i.MX8M based Linux development kit and an Edge TPU Accelerator USB dongle that can fit into any Linux computer including the Pi.
The Edge TPU is a lightweight, embedded version of its enterprise focused Cloud Tensor Processing Unit (Cloud TPU) AI co-processor. In conjunction with a new Cloud IoT Edge stack, the chip is designed to run TensorFlow Lite ML models on Arm Linux- or Android Things based IoT gateways connected to Google Cloud services.
Nvidia launches industrial TX2i and octa-core Xavier Jetson modules
Nvidia is farther along in its attempt to bring its Pascal/CUDA-related AI technologies to embedded Linux developers. Its Jetson TX1 and TX2 computer-on-modules have found widespread adoption in embedded Linux projects for ML applications. The Jetson TX2 recently appeared in devices including Axiomtek’s eBOX560-900-FL box computer, as well as an upcoming, FPGA-equipped AIR-T Mini-ITX board for AI-enabled SDR applications.
In recent months, Nvidia has begun shipping a Jetson TX2i spin on the TX2 aimed at industrial applications. The TX2i adds -40 to 85°C support, vibration resistance, and a wider humidity range. There’s also support for ECC RAM, a 10-year operating supply lifecycle, and a 3-year warranty.
Like the Jetson TX2, the TX2i provides dual high-end Denver 2 Arm cores, a quad-core, Cortex-A57 block, and a 256-core Pascal GPU with CUDA libraries for running AI and ML algorithms. Like the TX2, the module also supplies 8GB of LPDDR4 RAM, 32GB of eMMC 5.1, and 802.11ac WiFi and Bluetooth.
Existing Jetson carrier boards work with the TX2i. Aetina just announced an ACE-N310 carrier for all the Jetson modules that matches the TX2i’s industrial temperature support and supports six simultaneous HD cameras.
The Jetson TX2 was recently joined by a more powerful new Jetson Xavier module. The Xavier core, which has already been used in Nvidia’s Drive PX Pegasus autonomous car computer board, features 8x ARMv8.2 cores and a high-end, 512-core Nvidia Volta GPU with tensor cores. It also provides 2x NVDLA deep learning engines and a 7-way VLIW vision chip. The Xavier ships with 16GB 256-bit LPDDR4 and 32GB eMMC 5.1.
Google and Nvidia are not alone in their campaigns to bring cloud AI analytics to the edge. For example, Intel’s Movidius 2 neural network accelerator chip is finding widespread adoption. Presumably, however, any future AIY Projects kits will replace the Movidius 2 with the Edge TPU.
Although Amazon has yet to reveal a neural accelerator of its own, it is perhaps still the leader in the larger race for IoT edge analytics due to the popularity of its AWS IoT stack and its AWS Greengrass software for local processing of cloud analytics software on Linux devices. Meanwhile, Microsoft is also targeting the IoT space with its Arm Linux based Azure Sphere distribution and IoT framework. Azure Sphere will initially target lower-power applications running on Cortex-A7 chips. Future versions, however, may be more robust and may include a homegrown AI component.
Join us at Open Source Summit + Embedded Linux Conference Europe in Edinburgh, UK on October 22-24, 2018, for 100+ sessions on Linux, Cloud, Containers, AI, Community, and more.
