or “A short story on proprietary serial protocols”.
So I’ve bought myself a used Cressi Nepto on kleinanzeigen. As soon as I discovered that this diving computer (DC) can also sync the dive data to a PC, I had to get the required adapter as well. Luckily there was also a used one on kleinanzeigen for half the regular price :)
After a very painful trip for roughly 2 weeks with DHL, I finally had my hands on the adapter.
Did you know that Linus created other really useful software besides the Linux kernel and Git? Subsurface – the diving log software which is part of this story – was also started by him.
Let’s install Subsurface! Btw. did I mention that I’m using Arch? … oh well the packaged version in the AUR is super old, so let’s create a new package from the AppImage subsurface-appimage.
The software provided by Cressi is the same as for the Cressi Goa, so this should work with Subsurface, shouldn’t it?
Bummer, it can’t download the data.
So let’s dive into some weird way to read data from a dive computer via some proprietary adapter! Sounds like fun ;)
I know Jef, the maintainer of libdivecomputer, from a previous job and I thought it’s nice to get back in contact with him and asked him for advice. Jef promptly came back to me with his ideas and suggestions, thanks for that!
First steps first: what does the vendor software do?
Windows VM inside VirtualBox
Let’s get a Windows 11 VM to install in VirtualBox. Apparently there’s a “free of charge for some weeks” VM of this Windows 11 “operating system” provided directly by Microsoft. At least I can run that without giving it a network interface…
Please make sure to run the VM while being in the vboxusers group, otherwise one can’t forward USB devices to the VM. This can cause a lot of frustration “why doesn’t the DC show up in the app?”.
The Cressi UCI (Underwater Computer Interface) software looks like it was last updated a long time ago, but hey “it works”^TM.
So let’s try this out.
Alright, I can import all data, sync the time, set the units to metric or imperial.
How does UCI communicate with the DC? Via an FTDI serial converter that is connected to an Infrared link, let’s call it also IrDA as everyone else also does so.
Jef proposed to capture the data on the Windows VM, but I thought “naah, I don’t want to touch this Windows thing more than necessary. There’s wireshark and it has USB support on Linux, this must work”.
Capture USB traffic
And it was indeed straight-forward.
Install wireshark. Make sure that you’re in the wireshark group.
Run sudo modprobe usbmon to load the USB probes.
Run lsusb and check to which USB Host controller (indicated by the Bus) your serial adapter is connected to.
Open up wireshark and start capturing of usbmon<USB Host controller>, in my case it was usbmon5.
Filter on ftdi-ft.if_a_tx_payload || ftdi-ft.if_a_rx_payload to see only transferred data between the devices.
Alright! We have support for the Cressi Goa in libdivecomputer, the Cressi software is the same for Nepto and Goa, so there can’t be a huge difference in the serial protocol of the two DCs.
But next, back to basics of how communication via a serial interface usually works.
Basic frame format
As all good engineers should have done at least once in their life, Cressi (or I guess most likely some engineering shop) did invent their own serial protocol.
There’s a sync start-sequence, and end-octet, which defines some kind of fixed frame format. A length field and a CRC to make sure that transferred data is not corrupted. Below are the basics in some notation I just came up with to document this hell while looking at raw octets…
NB: All values are hex values, multi-octet types are little-endian.
SYNC_START = 0xaa aa aa
SYNC_END = 0x55
LEN = u8
CMD = u8
PAYLOAD = LEN * u8
MESSAGE = LEN | CMD [ | PAYLOAD]
CRC16 = crc16_ccitt(MESSAGE)
FRAME = SYNC_START | MESSAGE | CRC16 | SYNC_END
CMD 0x00 - Identity
To get the DC’s identity, the command 0x00 is sent and a reply is received, which contains the serial number, model, and firmware version of the DC.
# request
CMD = 0x00
# response
LEN = 0x0b => 11
CMD = 0x21
SERIAL = u32
MODEL = u8
FIRMWARE = u16
UNKNOWN_00 = 8 * u8
PAYLOAD = SERIAL | MODEL | FIRMWARE | UNKNOWN_0
# example data
aaaaaa0b21b21100000acc00010004004a7855
# PAYLOAD
b21100000acc0001000400
SERIAL = 0xb2110000 => 0x11b2 = 4530
MODEL = 0x0a => 10
FIRMWARE = 0xcc00 => 0x00cc = 204
UNKNOWN_00 = 0x01000400
Those values are consistent to what is shown on the DC – Hooray!
The length that is sent by the Nepto is different to what is expected from the Goa.
So let’s patch libdivecomputer to support the longer answer … oh nice, that was easy, now let’s see what else differs …
Bulk data retrieval
Let’s compare the capture with the sources of libdivecomputer and find out how the “real” data transmission works.
So there’s this request-response mechanism where each frame starts with 0xAAAAAA, but there is another mechanism for bulk data transmission.
There are commands like the logbook retrieval, which trigger the DC to send bulk data afterwards. That’s what libdivecomputer handles in cressi_goa_device_download()
That frame format looks as follows and it’s always 512 octets data + 5 octets header&footer.
ID = u8
FRAME_NO = u8
FRAME_NO_INV = u8
DATA = 512 * u8
CRC16 = crc16_ccitt(DATA)
The first frame, which starts with the header 0x0101fe, contains the real length of the data in its first 2 DATA octets.
In case the real length in a frame is shorter than 512 octects, it will be padded with 0xff.
# an example with 10 octets payload
ID = 0x01
FRAME_NO = 0x01
FRAME_NO_INV = 0xfe
DATA = 0x0a00<10 octets data><500 * 0xff>
CRC16 = crc16_ccitt(DATA)
Once the PC software received such a bulk data frame, it will send a single ACK octet of the value 0x06 to the DC.
The DC will then either send more bulk data or send an END octet of the value 0x04.
CMD 0x23 - Logbook
While comparing the sources for the Goa and the Wireshark capture one can see that the command code to read the logbook differs … the Goa expects 0x21, but the Nepto wants 0x23.
*sigh*
OK, it’s not that straight forward as hoped …
… and there is also payload contained in the initial response now … that wasn’t the case before.
# request
CMD = 0x23
## example
aaaaaa0023011455
# response
LEN = 0x02
CMD = 0x21
PAYLOAD = u16(0x2000) => 0x0020 = 32
## example
aaaaaa02212000785a55
Having a look at the Goa implementation and what the DC shows on its display, this indicates that we’re told now how many logbook entries exist.
CMD 23 - Logbook (cont’d)
After having given the data a looong good stare we’ll find patterns for sure!
# example data
0101fee001010005e5070c060d0901000e040000020005e5070c070b08010081060000030005e5070c080d26010014010000040005e5070c080d3601007c
000000050005e5070c080e04010083020000060005e5070c0a070b01007b040000070005e5070c0d10350100e2050000080005e5070c0d12220100fb0700
00090005e5070c1b121d0100cf0900000a0005e607010311230100be0c00000b0005e60704120a1601004d0000000c0005e6070502120801009b0500000d
0005e607050213220100980100000e0005e6070506111d0100e00500000f0005e607050b112a010051000000100005e607050b112c010067000000110005
e6070512123801003d000000120005e6070512123b01004a020000130005e607060f12190100840a0000140005e60706111222010064050000150005e607
061b11200100f7100000160005e607070612050100db0a0000170005e607070b112d0100d1040000180005e607070b12270100aa050000190005e6070716
121901004c0500001a0005e607080a122f0100990700001b0005e6070812112401008a0000001c0005e6070812112b0100d40000001d0005e60708131217
0100240a00001e0005e607081c0b2a01001d0500001f0005e607081c0e3601003b050000200005e8070409131a01002a040000ffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffa685
Header
Let’s apply what we’ve learned before. It’s a bulk data frame, so start to parse it like that …
0101fee001
ID = 0x01
FRAME_NO = 0x01
FRAME_NO_INV = 0xfe
LEN = u16(e001) => 0x1e0 = 480
480 octets … divided by 32 entries … looks like each entry is 15 octets long.
DATE format
In the next part of the payload seems to be a date included in the data.
The format is already implemented in libdivecomputer. For the sake of completeness, it looks as follows:
DATE_YEAR = u16
DATE_MONTH = u8
DATE_DAY = u8
DATE_HOUR = u8
DATE_MINUTE = u8
DATE = DATE_YEAR | DATE_MONTH | DATE_DAY | DATE_HOUR | DATE_MINUTE
Entries
LOG_ENTRY_ID = u16
DIVE_TYPE = u8 -- 0x05 = Freedive
START = DATE
UNKNOWN_DL_ENTRY_10 = u16
DIVELOG_SIZE = u32
LOG_ENTRY = LOG_ENTRY_ID | DIVE_TYPE | START | UNKNOWN_DL_ENTRY_10 | DIVELOG_SIZE
0100 05 e5070c060d09 0100 0e040000
[...]
2000 05 e8070409131a 0100 2a040000
CMD 0x22 - Dive
After having a look at the capture again, the next request is for each dive/logbook entry.
# request
CMD = 0x22
DIVE_ID = u16
PAYLOAD = DIVE_ID
## example
aaaaaa02220100ff3655
The returned direct response contains no data and the DC starts sending bulk data.
# start of the bulk data
0101fe 0e04 0100 ab01 e5070c060d09 02010c120c006700ab006e109e012900f5c4020085008940854089408940894081405d40594041402940c6052d002d
BULK_HEADER = 0x0101fe
LEN = u16(0e04) => 0x1e0 = 1038 -- nice, that's the same as last field in the logbook entry
DIVE_ID = u16(0100) -- cool, that's the dive id we just requested
UNKNOWN_22_07_U16 = u16(ab01) => 0x1ab = 427 -- ?
UNKNOWN_22_07_U8 = u8(ab) => 0xab = 171 -- ?
UNKNOWN_22_08_U8 = u8(01) => 0x01 = 1 -- ?
START = DATE(e5070c060d09) -- also cool, that's the same date as in the logbook entry
How does libdivecomputer handle the returned dive for the Goa … OK, those are 2 octet-wide sample values … and we have 1038 - 2 - 2 - 6 = 1028 remaining data octets.
How well does any of 427 or 171 and 2 octet-wide values play together with 1028?
The long stare technique (LST) doesn’t seem to help here, even after looking for a long time and trying different things … let’s go back to UCI and check what it does and provides!
UCI internals
UCI created a sqlite database (DB) with all the data it read from the DC.
There’s a table AdvancedFreeProfilePoint with a column Sequence that goes from 0 ... 426 … that’s 427 entries – cool! – (maybe those are the samples?)
So 427 * 2 = 854 … 1028 - 854 = 174, what are those remaining 174 octects and where do the samples start? 174 octets for a header? Sounds a bit much!?
Even with this information, the LST didn’t help … I also couldn’t correlate any of the values in the DB to that data.
Well, we have the installed application … let’s look a bit deeper into it!
UCI is a .NET application based on Unity, which is … a 3D gaming engine!? … hey I’m not gonna judge, it’s software and as long as you’re comfortable with it and it does the job, why not?
… at least that could maybe explain why I had to disable 3D acceleration for the VM, otherwise UCI wouldn’t even start.
There exist decompilers for .NET applications … so let’s fire up ILSpy, resp. its GUI version AvaloniaILSpy.
UCI decompiled
OK, let’s look into what got installed … some DLL’s … there’s Cressi.Algo.dll, but decompiling that shows nothing of interest.
At one point while digging through the intertubes, some SO answer (which I apparently can’t find again) mentions that all Unity C# code goes into Assembly-CSharp.dll and *BOOM*.
Here we go, with Cressi.IrdaApi, Cressi.IrdaApi.Data, IrdaApiV0 up to IrdaApiV4 and some other interesting stuff. No more LST required, let’s just look at the code.
Luckily the staring without results and trying stuff out didn’t take more than 2 or 3 hours in total, otherwise I’d feel even more stupid to not focus on this earlier :-D
CMD 0x22 - Dive (cont’d)
So the new IrdaApiV4 Dive data contains more data than the old version.
There’s an extended header and also some advanced dip statistics. As the Nepto is a DC mainly for Freediving it already differentiates between a Dive – total time spent in water – and a Dip – each time you go under water, usually multiple times during one dive.
Header
The Dive data has the following header:
0100ab01e5070c060d0902010c120c006700ab006e109e012900f5c4
DIVE_ID = u16 -- 0100
SAMPLES = u16 -- ab01 => 0x1ab = 427
START = DATE -- e5070c060d09
RATE = u8 -- 02
TARAVANA_FACTOR = u8 -- 01
SESSIONTIME = u16 -- 0c12
DIPS = u8 -- 0c => 12
UNUSED = u8 -- 00
MAXDEPTH = u16 -- 6700
MINTEMP = u16 -- ab00
SURFTIME = u16 -- 6e10
DIVETIME = u16 -- 9e01
BESTDIP = u16 -- 2900
CRC = u16 -- f5c4
Sample points
The header is followed by the sample points, which are still in the same format as with the Goa:
POINTS = SAMPLES * u16
020085008940...
Advanced dip statistics
The sample points are followed by the advanced dip statistics:
DIPTIME = u16
SPEED_DESC = u16
SPEED_ASC = u16
TARAVANA_VIOLATION = u8
DIP_DETAIL = SURFTIME | MAXDEPTH | DIPTIME | MINTEMP | SPEED_DESC | SPEED_ASC | TARAVANA_VIOLATION
DIP_DETAILS = DIPS * DIP_DETAIL
Subsurface doesn’t know about such things as advanced statistics, so we simply have to skip this data. There’s no way to tell Subsurface about the max ascend or descend speed, which would be kind of interesting, but then it’s also not that important. … yet … ;)
CMD 0x13 - Sync time
While trying out UCI and capturing the USB traffic there was also the option to synchronize the time on the DC with the PC.
Subsurface also has the option to do that, so I’d like to have that as well.
That one was straight-forward already from the dump.
LEN = 7
CMD = 0x13
SECONDS = u8
PAYLOAD = DATE | SECONDS
CMD 0x1D - Exit PC Link
Looking at the decompiled UCI, then looking at libdivecomputer which missed the close() API for the Goa, I decided to link close() to the “Exit PC Link” command.
LEN = 0
CMD = 0x1D
Some other details
Sorry, some of the field names I only found out either after looking at the DB or the decompiled code, but I can’t remember now which ones, so I just leave it as it is now :)
E.g. the DIVE_TYPE in the logbook entry made only sense once I looked at the DB.
Or the UNKNOWN_DL_ENTRY_10 should be called VERSION, but it also doesn’t matter, since it isn’t used.
Conclusion
Roughly two days after starting to dig into this topic I created libdivecomputer/libdivecomputer#33
Let’s hope this gets merged soon, so it can be included in one of the next Subsurface releases.
The import to Subsurface works, so I’m happy for now :)
EOF