Read AirTag data from the FindMy.app cache and convert to GPX

airtag-to-gpx-sync.sh

#!/usr/bin/env bash

# 
# Reads AirTag data from the FindMy.app cache and converts it to a daily GPX file
#
# Rsyncs the data to a web accessible folder that can be displayed with e.g. 
# https://gist.github.com/henrik242/84ad80dd2170385fe819df1d40224cc4
#
# This should typically be run as a cron job
#

set -o pipefail -o nounset -o errexit

export PATH=/usr/local/bin:$PATH
DATADIR=/tmp/airtag-data
TODAY=$(date +%d)

mkdir -p $DATADIR
DATA=$DATADIR/airtagdata-$TODAY.txt
GPX=$DATADIR/airtagdata-$TODAY.gpx
TAGNAME=Foobar

if [[ $(uname -s) == "Darwin" ]]; then
  TOMORROW=$(date -v +1d +%d)
else
  TOMORROW=$(date --date="tomorrow" +%d)
fi
rm -f $DATADIR/airtagdata-$TOMORROW.gpx

jq -r '.[] | select(.name == "'$TAGNAME'") | .location | "\(.latitude) \(.longitude) \(.altitude) \(.timeStamp/1000 | todate)"' \
   $HOME/Library/Caches/com.apple.findmy.fmipcore/Items.data >> $DATA

START='<?xml version="1.0" encoding="UTF-8"?>
<gpx xmlns="http://www.topografix.com/GPX/1/1" xmlns:mytracks="http://mytracks.stichling.info/myTracksGPX/1/0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" creator="myTracks" version="1.1" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd">
   <trk>
      <name>'$TAGNAME'</name>
      <extensions>
         <mytracks:color red="0.000000" green="0.000000" blue="1.000000" alpha="1.000000" />
         <mytracks:area showArea="no" areaDistance="0.000000" />
         <mytracks:directionArrows showDirectionArrows="yes" />
         <mytracks:sync syncPhotosOniPhone="no" />
         <mytracks:timezone offset="120" />
      </extensions>
      <trkseg>'

END='      </trkseg>
   </trk>
</gpx>'


echo $START > $GPX 

function elems() {
    LAT=$1
    LON=$2
    ELE=$3
    TS=$4
}

cat $DATA | while read line; do
  elems $line
  echo '<trkpt lat="'$LAT'" lon="'$LON'">
            <ele>'$ELE'</ele>
            <time>'$TS'</time>
         </trkpt>' >> $GPX
done

echo $END >> $GPX
cp $GPX $DATADIR/airtagdata.gpx

rsync -a --exclude='*.txt' $DATADIR example.com:public_html/airtag/

@wilkyconsultants @henrik242 @varenc

Note: This is not as easy in macOS 15+, since the items in ~/Library/Caches/com.apple.findmy.fmipcore/ are now encoded. You can get them more readable in XML with sudo plutil -convert xml1 -o ~/Downloads/Items.txt ./Items.data but it's still an unknown.

And ask Gemini/Ollama/GPT-4 etc about using Darwin's security and plutil. I was able to find my apple user ID and password, which are then Base64 encoded for Basic Auth, in com.apple.account.AppleAccount.cloudkit-token.

Also, try binwalk a copy of the /usr/libexec/fmfd (find my friends daemon) with binwalk --dd='.*' fmfd to extract all to a folder in the same directory, prefixed by an underscore.

Anyone good at reading hex and byte offsets can make work which certs it's using. Example output without the --dd:

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             Mach-O universal binary with 2 architectures
20364         0x4F8C          Unix path: /usr/lib/dyld
24456         0x5F88          Unix path: /usr/lib/swift/libswiftObjectiveC.dylib
647500        0x9E14C         XML document, version: "1.0"
778255        0xBE00F         mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
857010        0xD13B2         XML document, version: "1.0"
863590        0xD2D66         Certificate in DER format (x509 v3), header length: 4, sequence length: 1038
864632        0xD3178         Certificate in DER format (x509 v3), header length: 4, sequence length: 1204
865840        0xD3630         Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
866515        0xD38D3         Certificate in DER format (x509 v3), header length: 4, sequence length: 260
867417        0xD3C59         XML document, version: "1.0"
889036        0xD90CC         Unix path: /usr/lib/dyld
893128        0xDA0C8         Unix path: /usr/lib/swift/libswiftObjectiveC.dylib
1563692       0x17DC2C        XML document, version: "1.0"
1728599       0x1A6057        mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
1758178       0x1AD3E2        XML document, version: "1.0"
1764758       0x1AED96        Certificate in DER format (x509 v3), header length: 4, sequence length: 1038
1765800       0x1AF1A8        Certificate in DER format (x509 v3), header length: 4, sequence length: 1204
1767008       0x1AF660        Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
1767683       0x1AF903        Certificate in DER format (x509 v3), header length: 4, sequence length: 260
1768585       0x1AFC89        XML document, version: "1.0"

This is a tad over my head but deepseek had some good info.

This is a fascinating and highly technical exploration of macOS internals, specifically related to Apple's Find My system, Airtags, and the security mechanisms in place. Let’s break this down and provide some guidance on how to proceed with the analysis, as well as some insights into the tools and techniques mentioned.

1. Understanding the Context
   You’re exploring the cache data and binaries related to Apple’s Find My system, including Airtags and Find My Friends. The goal seems to be decoding or extracting useful information from these files, such as:

Decoding the Items.data cache file in macOS 15+.

Extracting Apple user IDs and passwords from com.apple.account.AppleAccount.cloudkit-token.

Analyzing the fmfd (Find My Friends Daemon) binary for certificates and other embedded data.

2. Decoding Items.data in macOS 15+
   As you mentioned, the Items.data file in ~/Library/Caches/com.apple.findmy.fmipcore/ is now encoded in macOS 15+. Here’s how you can proceed:

Using plutil to Convert to XML
You’ve already identified the correct command to convert the binary .data file to XML:

sudo plutil -convert xml1 -o ~/Downloads/Items.txt ./Items.data
This will give you a more readable XML file, but the structure and content might still be cryptic. You’ll need to analyze the XML to understand the data structure.

Next Steps
Analyze the XML: Look for patterns or identifiable data (e.g., device IDs, timestamps, locations).

Compare with Older Versions: If you have access to an older macOS version, compare the XML structure to see what has changed.

Automate Parsing: Write a script (e.g., in Python) to parse the XML and extract meaningful data.

3. Extracting Apple User ID and Password
   You mentioned finding your Apple user ID and password in com.apple.account.AppleAccount.cloudkit-token, which is Base64 encoded for Basic Auth. Here’s how to proceed:

Steps to Decode
Locate the File: The file is likely in ~/Library/Accounts/ or a similar directory.

Extract the Token: Use a tool like base64 to decode the token:

echo "BASE64_ENCODED_STRING" | base64 --decode
Analyze the Output: The decoded string should contain your Apple ID and password (or a token). Be cautious with this data, as it’s sensitive.

Security Considerations
Keychain Access: macOS stores sensitive data in the Keychain. You might need to use the security command to access it:

security find-generic-password -a "AppleAccount" -s "cloudkit-token"
Permissions: You’ll need appropriate permissions (e.g., sudo) to access these files.

4. Analyzing fmfd with binwalk
   You’ve already started analyzing the fmfd binary using binwalk. Here’s how to proceed:

Extracting Embedded Data
Use the --dd option to extract all embedded files:

binwalk --dd='.*' /usr/libexec/fmfd
This will create a folder with extracted files, prefixed by an underscore.

Analyzing Certificates
The binwalk output shows several certificates in DER format. You can use openssl to decode these:

openssl x509 -inform der -in extracted_certificate.der -text
Look for details like issuer, subject, and validity periods to understand their purpose.

Hex and Byte Offset Analysis
Use a hex editor (e.g., xxd, hexdump, or a GUI tool like Hex Fiend) to inspect the binary at specific offsets.

Look for patterns, strings, or headers that indicate embedded data (e.g., XML, certificates, keys).

5. Tools and Techniques
   Here’s a summary of the tools and techniques you’ll need:

plutil: For converting binary property lists to XML.

base64: For decoding Base64-encoded strings.

security: For accessing macOS Keychain data.

binwalk: For analyzing and extracting embedded data from binaries.

openssl: For decoding and inspecting certificates.

Hex Editors: For manual analysis of binary files.

6. Ethical and Legal Considerations
   Privacy: Be mindful of the sensitive nature of the data you’re working with (e.g., Apple IDs, passwords, certificates).

Permissions: Ensure you have the right to access and analyze these files, especially on systems you don’t own.

Compliance: Respect Apple’s terms of service and applicable laws when reverse-engineering their software.

7. Next Steps
   Automate the Process: Write scripts to automate the extraction and decoding of data.

Document Findings: Keep detailed notes on the structure and content of the files you analyze.

Share Knowledge: If you discover something useful, consider sharing it with the community (while respecting privacy and legal boundaries).

This is a complex but rewarding project that combines reverse engineering, macOS internals, and security analysis. Let me know if you’d like help with specific steps or tools!

@wilkyconsultants

I honestly wouldnt be surprised if deepseek used my posts on other repos Issues and gists. some of those sentences I've said verbatim. See here.

Some good additional insights too. I'll keep investigating.

‼️ ALERT ‼️

If you have a second mac, turn off automatic updates and keep on 14. Sequoia's Image Playground is crap and eats up so much space anyway. Pretty 'meh' release.

I’ve been following this for a while, and the best solution I found is to buy a $100 Mac and keep it on the last macOS version before Apple started encrypting the Find My cache files. I use it as a dedicated server, running 24/7, to continuously execute the script. That script updates a directory on my iCloud Drive, which syncs seamlessly with my other devices. I then built a very simple iOS app that retrieves and parses this data from the synced folder. The app allows me to select a time and date, and it returns the location history for that period on AppleMaps using the GPX structure generated by the script.

I’d love to add a couple of features to my app, like a heatmap and a paper trail, but I’m still new to Swift. I’ve been following others who are trying to decode the data through reverse engineering, and it seems like Apple is reinforcing the encryption with every update. It’s while to me—this is your own data.....................

I’ve been following this for a while, and the best solution I found is to buy a $100 Mac and keep it on the last macOS version before Apple started encrypting the Find My cache files. I use it as a dedicated server, running 24/7, to continuously execute the script. That script updates a directory on my iCloud Drive, which syncs seamlessly with my other devices. I then built a very simple iOS app that retrieves and parses this data from the synced folder. The app allows me to select a time and date, and it returns the location history for that period on AppleMaps using the GPX structure generated by the script.

I’d love to add a couple of features to my app, like a heatmap and a paper trail, but I’m still new to Swift. I’ve been following others who are trying to decode the data through reverse engineering, and it seems like Apple is reinforcing the encryption with every update. It’s while to me—this is your own data.....................

Yea your right, I have been working on decrypting sequoia Items.data for a while now and can't see how to get the decryption key so giving up for now. I decided to build a bunch of VMs running older Macos version for now (Monterey works great, Sonoma is a bit of a challenge under VirtualBox). Have 5 VMs running under VirtualBox on a Windows PC (16GB memory 2.3Ghz cpu). I am harvesting data on 160 airtags which I feed in to my django server and serve the data via Rest API to my iOS and android app clients. Works well so far. Saves having to buy and maintain a bunch of Macs!

@wilkyconsultants - which setup to do use to create the VM?

@wilkyconsultants - which setup to do use to create the VM?

Most Monterey ISO's on archive.org works fine with VirtualBox with default set up. If anyone manages to get past a boot loop panic on Sonoma 14.3.1 in VirtualBox please let me know. Tried everything..

The best recommendation is to buy a cheap Mac that supports macOS Sonoma 14.3.1 (minimum models: MacBook Air 2018, MacBook Pro 2018, Mac mini 2018, iMac 2019, or newer).
Install macOS Sonoma 14.3.1, the last version before Apple started encrypting Find My cache files.
This allows you to read AirTag and shared item locations directly from:

~/Library/Caches/com.apple.findmy.fmipcore/Items.data
~/Library/Caches/com.apple.findmy.fmipcore/Devices.data

With this setup, you can still extract the data in plain format using available scripts.
In newer macOS versions, reading this data is practically impossible due to the encryption implemented by Apple.

The best recommendation is to buy a cheap Mac that supports macOS Sonoma 14.3.1 (minimum models: MacBook Air 2018, MacBook Pro 2018, Mac mini 2018, iMac 2019, or newer). Install macOS Sonoma 14.3.1, the last version before Apple started encrypting Find My cache files. This allows you to read AirTag and shared item locations directly from:

~/Library/Caches/com.apple.findmy.fmipcore/Items.data
~/Library/Caches/com.apple.findmy.fmipcore/Devices.data

With this setup, you can still extract the data in plain format using available scripts. In newer macOS versions, reading this data is practically impossible due to the encryption implemented by Apple.

I am using a Mac Mini 2012 with Sonoma 14.3.1 for my data collection along with a bunch of VMs running Monterey. Sonoma is better as it can access shared tags as well but Monterey is a good fall back since it can run in a small Macos VM under windows. With a 16 GB memory on a windows box you can run 5 VMs with no issues getting you 5 x 32 tags = 160 on 1 box!

Hello, I want to reinstall a Mac mini with 14.3.1. Unfortunately, I can only find 14.7.4.

If I understand correctly, 14.7.4 already has encryption enabled.

Does anyone have a link to 14.3 or 14.3.1?

I would, of course, like to install the version that provides the most data, including AirTags and shared data.

Hello, I want to reinstall a Mac mini with 14.3.1. Unfortunately, I can only find 14.7.4.

If I understand correctly, 14.7.4 already has encryption enabled.

Does anyone have a link to 14.3 or 14.3.1?

I would, of course, like to install the version that provides the most data, including AirTags and shared data.

Check https://archive.org for Sonoma 14.3.1, let me know if you need help, built some pretty awesome code for FindMy harvesting.

The web archive worked. Thank you.

I'm interested in the code. I'd also like to share my code. What do I do? Do we do this here on GitHub? Directly here?

You can just fork this gist :)

I looked at this a while ago and I believe the encryption key for the Items/Devices.data binary plists may be this item in iCloud Keychain. Keychain Access won't display the value, more research needed on how to dump this:

The Find My app on Mac loads these dylibs and there is code related to decrypting the plist files in them. I got as far as setting breakpoints around where it's doing the decryption but haven't extracted the keys yet.

You can extract the dylibs from the macOS dylib cache using ipsw:

% ipsw dyld extract /System/Volumes/Preboot/Cryptexes/OS/System/Library/dyld/dyld_shared_cache_arm64e FindMyCrypto FMFCore FMIPCore -o .

% file *
FindMyCrypto: Mach-O 64-bit dynamically linked shared library arm64e
FMFCore: Mach-O 64-bit dynamically linked shared library arm64e
FMIPCore: Mach-O 64-bit dynamically linked shared library arm64e

Process 33543 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = instruction step over
    frame #0: 0x00007ffb24c14a1a FindMyCrypto`FindMyCrypto.SymmetricKey256.init(from: Swift.Decoder) throws -> FindMyCrypto.SymmetricKey256 + 42
FindMyCrypto`FindMyCrypto.SymmetricKey256.init(from: Swift.Decoder) throws -> FindMyCrypto.SymmetricKey256:
->  0x7ffb24c14a1a <+42>: movq   -0x8(%rax), %rax
    0x7ffb24c14a1e <+46>: movq   %rax, -0x38(%rbp)
    0x7ffb24c14a22 <+50>: movq   0x40(%rax), %rax
    0x7ffb24c14a26 <+54>: callq  0x7ffb24c01630            ; __chkstk_darwin
Target 0: (FindMy) stopped.

* thread #1, queue = 'com.apple.findmy.fmcore.diskio', stop reason = breakpoint 62.1
    frame #0: 0x00007ffb24c05c20 FindMyCrypto`FindMyCrypto.decryptAndValidate(envelope: FindMyCrypto.EncryptAndSignEnvelope, symmetricKey: FindMyCrypto.SymmetricKey256, publicKey: FindMyCrypto.P256PublicKey) throws -> Foundation.Data
FindMyCrypto`FindMyCrypto.decryptAndValidate(envelope: FindMyCrypto.EncryptAndSignEnvelope, symmetricKey: FindMyCrypto.SymmetricKey256, publicKey: FindMyCrypto.P256PublicKey) throws -> Foundation.Data:
->  0x7ffb24c05c20 <+0>: pushq  %rbp
    0x7ffb24c05c21 <+1>: movq   %rsp, %rbp
    0x7ffb24c05c24 <+4>: pushq  %r15
    0x7ffb24c05c26 <+6>: pushq  %r14

Yea your right, I have been working on decrypting sequoia Items.data for a while now and can't see how to get the decryption key so giving up for now. I decided to build a bunch of VMs running older Macos version for now (Monterey works great, Sonoma is a bit of a challenge under VirtualBox). Have 5 VMs running under VirtualBox on a Windows PC (16GB memory 2.3Ghz cpu). I am harvesting data on 160 airtags which I feed in to my django server and serve the data via Rest API to my iOS and android app clients. Works well so far. Saves having to buy and maintain a bunch of Macs!

Hi @wilkyconsultants,

Your setup with VirtualBox VMs running Monterey on Windows to collect AirTag data and serve it via Django is exactly what I’d like to replicate.

Would you mind sharing a few more technical details?
• Which version of Windows are you using as the host?
• What are the exact VirtualBox VM settings you used (CPU, RAM, storage, EFI, network, etc.) to get Monterey running stably?
• Which Monterey version worked best for you?
• Any specific steps or advice for enabling and using Find My in the VM, and making sure it works with your Apple ID?
• Did you use a prebuilt macOS image, or build it yourself? If you have a link or reference to the image or guide you used, it would help a lot.
• Any tricky parts or tips you wish you knew before starting?

Any info or resources you can share would be incredibly helpful. Thanks again for sharing your results!

Yea your right, I have been working on decrypting sequoia Items.data for a while now and can't see how to get the decryption key so giving up for now. I decided to build a bunch of VMs running older Macos version for now (Monterey works great, Sonoma is a bit of a challenge under VirtualBox). Have 5 VMs running under VirtualBox on a Windows PC (16GB memory 2.3Ghz cpu). I am harvesting data on 160 airtags which I feed in to my django server and serve the data via Rest API to my iOS and android app clients. Works well so far. Saves having to buy and maintain a bunch of Macs!

Hi @wilkyconsultants,

Your setup with VirtualBox VMs running Monterey on Windows to collect AirTag data and serve it via Django is exactly what I’d like to replicate.

Would you mind sharing a few more technical details? • Which version of Windows are you using as the host? • What are the exact VirtualBox VM settings you used (CPU, RAM, storage, EFI, network, etc.) to get Monterey running stably? • Which Monterey version worked best for you? • Any specific steps or advice for enabling and using Find My in the VM, and making sure it works with your Apple ID? • Did you use a prebuilt macOS image, or build it yourself? If you have a link or reference to the image or guide you used, it would help a lot. • Any tricky parts or tips you wish you knew before starting?

Any info or resources you can share would be incredibly helpful. Thanks again for sharing your results!

happy to share all but can't dump my versions of software until jul 4 cuz i am at my cottage til then and servers are at home, didn't set up remote access so don't recall versions.attached is my ecosystem for Mr Tracker (my app for airtags).

happy to share all but can't dump my versions of software until jul 4 cuz i am at my cottage til then and servers are at home, didn't set up remote access so don't recall versions.attached is my ecosystem for Mr Tracker (my app for airtags).

Thanks so much for being willing to share your setup—your info will definitely help me and others!

Your diagram is very similar to what I have, but I’m still limited by physical Mac minis, so I can’t scale like you do with VMs. Any details you share when you’re back will be super helpful.

Really appreciate your openness!