Developing an embedded system for remote field duty is hard enough, but what if you had to contend with -40ºC temperatures, high winds, ice-encased cables, and attacks from Arctic wildlife? These are just some of the harsh realities faced by the developers of a Linux-driven sensor buoy deployed on the sea ice off the north coast of Alaska.
At the recent Embedded Linux Conference Europe (ELCE), Satish Chetty talked about his volunteer work setting up a sea ice monitoring station funded by Ice911. The principal goal is to study changes in ice formation and melting due to global warming. Chetty’s day job is VP of software engineering at Hera Systems, a Silicon Valley startup that develops Earth imaging satellites and edge analytics solutions.
The mostly autonomous monitoring buoy has been evolving since 2009. Planted in or near sea ice from November to July every year, the station measures weather, water temperature, water depth (sonar), ice depth and melt, sunlight, and albedo (the reflection of sunlight). Cameras are used for visual analysis.
A custom, multi sensor, 1-Wire temperature string is attached to the buoy and embedded into the ice, “with sensors at every depth so you get a profile of water and ice thickness,” said Chetty. “Where we were testing, most of the melt happens from the bottom up because the meltwater flows into the water, heating it up.”
Like the underwater, Linux-driven ESP monitoring station described by Brent Roman at a presentation at last year’s ELCE conference, Chetty’s Arctic buoy is severely restrained by power. The site is just off the Arctic Ocean coast near Barrow, the northernmost town in the United States. The location sits in darkness for 65 days of the year, and even in warmer months, a battery bank is required to augment the solar panels.
Four panels are positioned at almost 90-degree angles to track the sun as it passes just over the horizon in a circular path. This configuration increases exposure to the fierce winds caused by the site’s peninsular location. As a result, Chetty’s team was forced to use small, 5-10 Watt panels so they wouldn’t blow over.
Originally, they used non-rechargeable Lithium batteries. For various reason, including the greater difficulty of replacement, as well as the regulatory hassles of transporting the batteries by air, they switched to banks of smartphone LiPo batteries. The developers and researchers are away from Barrow, so regular maintenance is typically performed by armed bear guards, who also accompany researchers during their visits to the buoy.
Wireless power hogs
The station’s biggest power draw comes from the cell modems, followed by multiple cameras. The station relies primarily on a $50 Huawei 3G cellular modem to transmit data to an archiving server. To avoid cellular service charges, the team originally started to set up WiFi repeaters, but abandoned the project due to the complexities of maintenance.
They did, however, add a WiFi access point, which is used for close-range communications with researchers’ mobile devices. “Sometimes the 3G and satellite modems fail so we have to go out and retrieve the SD card,” explained Chetty. “During melting, the buoy is surrounded by slushy, dangerous water, so we had to put a board down to reach it. It was hard pulling an SD card wearing gloves while balancing on a board. It’s much easier to use WiFi.”
Chetty and his team chose WiFi over Bluetooth to ease simultaneous access by multiple researchers. Yet, WiFi added other challenges. “Certain WiFi drivers require other network drivers before you can compile, so it adds to the complexity and boot time, and it burns more power,” said Chetty.
Power efficiency was the main consideration in system design, followed by cost, size, and weight. “The equipment needs to be small and light enough to be carried by an ATV or a sled pulled by snowmobiles, and so it can be easily dragged into a boat in July,” said Chetty. The system was also designed so it could be quickly disassembled. “The ice melt happens within a single week so you want to be able to quickly disassemble it,” said Chetty.
The station runs Linux on a Technologic TS-7400-v2 SBC connected to a Belkin USB hub. Chetty’s team considered using a cheaper and more power efficient microcontroller-based system, but selected Linux for several reasons. One was that most of the sensors they wanted to use were low-cost off the shelf devices with USB drivers. “Instead of making custom PCBs, it was easier to use a Linux system and just plug in the sensors.” Chetty developed a custom kernel for the board with a Debian stack that was trimmed to reduce non-essential packages.
Chetty praised the TS-7400-v2 for its $150 price, fanless operation, power efficiency, and -40 to 85ºC range. The ARM9-based i.MX286 SoC can be configured down to 454MHz to save on consumption. “The SBC can run at half a watt, and it can operate at 8 to 24V power, which is good because the battery doesn’t maintain charge all the time,” said Chetty. “There’s a built-in sleep timer that you can program to shut off after doing tasks, and we can turn peripherals on and off via software.”
The board includes a Real Time Clock (RTC), but at extremely low temperatures, it slows down causing time synchronization issues. “Every three or four days we do a time sync,” said Chetty. Originally, the developers performed remote config updates using ssh, but now they update once a year during the summer.
Prepping for cold, ice, and polar bears
Unlike most industrial systems, the station experiences -40ºC temperatures on a regular basis. The SBC works fine at -40ºC, as do the $75, USB-connected Logitech webcams, said Chetty. “Our 3G modem is rated only for -20ºC,” he added. “Lower than that it still connects, but it occasionally drops connections during handshakes.” In that case, sensor data is stored on the SD card.
Ice buildup proved to be a bigger challenge than low temperatures. For example, the Logitech cameras are housed in a fishing bait box that resists ice build-up, but still allows icicles to extend into the camera’s cutout view. When the cameras grabbed stills from the video, they focused on the icicles instead of the landscape.
Chetty’s solution was to run video capture for 3-5 seconds before taking the still, giving the cameras time to refocus. The system could then identify the good stills to save while discarding the remaining video to save disk space. “Compiling that at the kernel level was important,” said Chetty.
Ice and rime buildup on cables and sensors was a bigger problem. “The sonar sensor got so much ice on it after every blizzard that we kept getting incorrect readings,” Said Chetty. “For a while, we sent people out to chip off the ice, but it happened so often we decided to change the sensor. Just because it’s temperature rated, doesn’t mean it can handle every situation. At -40ºC, cables get encased in ice and can get brittle, and you tap them you can break them. The ice makes it hard to open the box up to repair things. One time we broke the board pins and ruined the experiment. We can’t take the whole thing back to the lab to fix it because the sensors are embedded into the ice. For our next version, we’ll put connectors outside instead of running cables inside.”
If all this wasn’t enough, there are also the animal attacks. “One time, a fox chewed out our sensors, so we put a cap on it,” said Chetty. “We think a polar bear stepped on one of the arms and broke some other sensors. When we see the sensor data acting weird, we know something has happened.”
You can watch the entire presentation below:
