Arsenal 2 March 2021 Update

Hey everyone, Ryan here.

The Arsenal team has been very heads down since our last update, focusing on the long list of deliverables as we work towards shipping Arsenal 2 in June. We’ve made some great progress, so let’s jump into some updates.

If a ton of detail isn’t your thing, and it has come to my attention that for many people it isn’t, here’s the executive summary: great progress, a few risks remain, our forecasted shipping date remains June.

Timeline

Most of the components necessary for Arsenal 2 have arrived. We’re short some CPUs for Arsenal 2 Standard and Pro, and a few other miscellaneous parts, but we have enough to complete the Pilot Runs this month. We expect the remainder of the necessary components to arrive between now and when we start Mass Production, so that shouldn’t impact the schedule.

The Arsenal 2 Standard Pilot Run is underway, which is very exciting! In the next update we’ll be able to share some photos of Pilot Run units we’ve received, and update on the progress there. We were quite happy with the Sample Run units (a very small number of units made before the Pilot Run) so are expecting good things here.

I talked a little about going to China and the COVID-related travel restrictions in the last update. Since then, China has increased their quarantine period to 21 days (from 14). Phillip (our electrical engineer) and I will need to be there to oversee the beginning of the Mass Production runs, so we’re gearing up to leave for China pretty soon. COVID and related travel restrictions have been a challenge from the start of the project, and remains a risk to the schedule. Also a risk is Phillip and I losing sanity while quarantined in a hotel room for 21 days.

Solving the Required Updates Problem

After the launch of Arsenal 1, we heard from users that they would sometimes be out in the field and the app would require a firmware update. To prevent needing to test different versions of the mobile app and firmware (something that quickly gets out of hand), Arsenal always requires the firmware and mobile app run the same version. Unfortunately in Arsenal 1, this meant if your mobile app had updated in the background and you didn’t have an internet connection, you would get stuck at a firmware update screen.

For Arsenal 2, we set out to solve this. To do so we did a complete rewrite of the firmware update system, and made some changes to shrink the firmware size. One big savings was a change to the way we stored/indexed the similar photos dataset. Previously it took up over 300MB on disk, and now it’s stored in about 50MB, with the same accuracy and faster lookup times. These changes, combined with a change to the way we build our operating system, means firmware updates now take less than 90 seconds start to finish (down from about 8 minutes for the install previously, plus however long it took to download).

In addition the firmware images are now small enough to fit in the app, so there is no download step when updating. So if your mobile app updates, you can update the firmware to match without needing an internet connection! To recap, the update process is faster, simpler, and eliminates any risk of needing to update without the internet.

Software Updates

Lately the software team and I have been focusing on a few different areas. The tl;dr is software is going well, still lots of debugging, user interface work, feature improvements, and test writing. Below is my usual level of nerdy detail if you’re curious.

Debugging

Lots of debugging and testing going on lately. We’ve been writing a lot of automated tests and improving our test rig (“the banana stand”) so we cover some more things we want to test. Each update here improves our testing reliability and speed, and is especially helpful for future app updates after we ship.

Crowd Control Performance

Arsenal 2’s Crowd Control is a fairly simple feature from a user standpoint, but the challenge is efficiently generating the final image (preview and full size) while it’s being shot (this lets you stop shooting as soon as the moving subjects have disappeared). The previous version used a lot more RAM and CPU time. After reading quite a few papers, I implemented a much faster core algorithm (in Halide) that lets us generate previews in less than half of our available RAM (without needing to reparse images - so it's super fast to run as you go). It can also generate full sized with a single pass over all images (in tiles).

Night Assist

Andrew has been finishing up the UI around Night Assist. There’s a lot of little things with how you switch into Night Mode, the feedback you get while it's focusing, etc. I should have a demo of the new UI in the next update.

Deep Color

The machine learning model that runs Deep Color isn’t that large compared to models that run in the cloud (cough GPT-3), but it’s the largest model we run on Arsenal 2. I’ve been reworking some of the model structure to improve inference speed on the device. Still a bit of work to go here, but it's looking really good so far.

Save Menu

Arsenal 2 has a lot of options for what files get saved where. Ben worked really hard to make what I think is a simple UI that lets you decide what files you want where.

Settings Calibrations

In the past, there were quite a few places where we had hardcoded tables for each camera so Arsenal knew how to jump between different settings. We’ve now replaced them with an automatic “calibration” routine that runs on our “banana stand” rig. It was a bit of work to get this up and running, but doing this in an automated fashion will prevent some of the issues we ran into with Arsenal 1.

Kernel/Hardware Tunings

There’s a lot of interesting choices regarding how a device runs when you design your own hardware. We’ve been making a lot of tweaks lately to how Arsenal 2 runs when the battery is almost dead, and when it’s on a charger that isn’t supplying much power. We think we’ve got a good setup now that does its best to keep things like timelapses running, even if it has to slow processing a bit (not an issue with most chargers nowadays, but enough that we should account for it).

Brightness Control

Lastly, by popular request (and by popular request I mean I wanted to use it myself), we made it so you can adjust the max brightness on the LED’s for night photography and such (instead of the previous on/off). This should help prevent light bleed if you still want some indicators.

Thanks to the Arsenal 2 team for all of the hard work. Arsenal 2’s software is really starting to come together and it’s been really exciting for me to start field testing with the latest.

Way Too Much Info On Arsenal 2 Runtime

Another tech note for the electronics enthusiasts out there. We know runtime is important, so we’ve spent a lot of time optimizing the power system for Arsenal 2. What we discovered is providing power to a high-performance processor with only 4 square inches of real estate to work with is a bit of a challenge.

In total, the circuitry for Arsenal 2 requires 13 unique power rails in order to function. Not only that, each power rail has its own set of requirements (voltage, maximum current, maximum noise, efficiency, etc.). Ensuring a long runtime means we’ve had to review each power rail, evaluate its current draw in several situations (for example; idle, live view off, live view on, photo-processing, etc.), and determine what topology works best. We’ve left no stone unturned, so we even evaluated low current power rails.

We had assumed there wouldn’t be much to improve on the low current power rails since they consume so little power to begin with, but we couldn’t have been more wrong. For example, early prototypes of the Arsenal 2 hardware used linear regulators for all the low current power rails. Linear regulators are a simple, low part count solution for converting a voltage to a lower voltage. But they are notoriously inefficient. But who cares about efficiency right? These are just the low current power rails.

Well, it turns out that by replacing just one of these linear regulators with a high efficiency DC-DC converter and reconfiguring power feeds for the remaining linear regulators, we can decrease the idle current draw from the battery by about 35mA. 35mA may not sound like much, but this decrease in the idle current draw INCREASES the idle runtime by about 45 minutes. When Phillip told me this, he picked up a mic in the office and dropped it. It was well deserved (and the mic is okay).

Wrap Up

Another long update, but I hope most of you enjoy the detail. My next update may be from China, which is a big step in the process of launching Arsenal 2. Thanks again for everyone’s support and more to come soon!

• Ryan

P.S. Here’s a few shots from recent field testing trips.