Skip to content

Instantly share code, notes, and snippets.

@nuclearsecrecy
Last active August 8, 2024 15:36
Show Gist options
  • Save nuclearsecrecy/2347187cf438ea3b0c9d93783c5f2963 to your computer and use it in GitHub Desktop.
Save nuclearsecrecy/2347187cf438ea3b0c9d93783c5f2963 to your computer and use it in GitHub Desktop.
Applescript function for finding specific UI elements in an application window by their name and (optionally) their class
# This is a function that will try to search a given set of elements for a
# matching class and name property. See above for example usage, where "Recognize..."
# is the name, `button` is the class (or `false` if you don't care about matching the class),
# and `entire contents` is the set of all UI elements to search. Sometimes elements don't have a
# name or class, which is why the try statement is in there.
# It returns "false" if the element was not found.
#
# This is not super fast, so it is probably best used to get a static reference,
# and not in a dynamic function. On my machine it takes maybe 4 seconds to search through
# 60 elements or so, 20ish seconds to search through 400 elements (not uncommon for a
# full window, if it can't find the element). If it was possible to cull targetElements
# to only the desired classes ahead of time, that would speed it up, but it doesn't
# seem like one can do that...
on getElementByClassAndName(targetName, targetClass, targetElements)
# log (count targetElements) # commented lines are just for debugging
# set i to 0
set found to false
repeat with uiElem in targetElements as list
if (targetClass is not false) then
try
if ((name of uiElem is targetName) and (class of uiElem is targetClass)) then
set found to true
end if
end try
else
try
if (name of uiElem is targetName) then
set found to true
end if
end try
end if
if found is true then
return uiElem
exit repeat
end if
#log (i)
#set i to i + 1
end repeat
return false
end getElementByClassAndName
# Example usage -- this assumes TextEdit is open and has a document open. Will find
# the path to the menu item with the name "Minimize" and log it to the console.
# Expected output is something like:
# (*menu item Minimize of menu Window of menu bar item Window of menu bar 1 of application process TextEdit*)
# Note that if you don't know the exact class, put in `false` instead of `menu item` and
# it'll return the first class with the matching name.
tell application "System Events"
tell process "TextEdit"
set searchElements to entire contents #note this is searching the entire process -- could change to `entire contents of front window`, for example, to narrow in scope
set myElement to my getElementByClassAndName("Minimize", menu item, searchElements)
log (myElement)
end tell
end tell
@nuclearsecrecy
Copy link
Author

This is an Applescript function that can be used to find UI elements dynamically in an application window. It is extremely slow (it can take 20-30 seconds to check every element in a window, if it has to, which is absurd), because the look up of the name or class is a very slow operation in Applescript, apparently. If I can figure out a way to speed it up, I'll update this, but for now, this is useful if you are trying to get the "path" to a given UI element in an application.

@tc0nn
Copy link

tc0nn commented Aug 8, 2024

Love it. Thanks for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment