Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save jakesOneFourSeven/cc8fab1e8712a34dbd596ebd355d76ea to your computer and use it in GitHub Desktop.

Select an option

Save jakesOneFourSeven/cc8fab1e8712a34dbd596ebd355d76ea to your computer and use it in GitHub Desktop.
Frigate - Telegram Notification
blueprint:
name: Frigate - Telegram Notification with Generative AI comments
description: Create automations to receive Snapshots and Clips from Frigate
domain: automation
input:
camera:
name: Frigate Camera
description: "Select the camera entity that will trigger notifications. \nIf you do not see cameras listed in the drop down, check you have the frigate integration installed.\nNote: The automation relies on this matching your frigate config (by default it does).\n"
selector:
entity:
integration: frigate
domain:
- camera
device_class:
- camera
multiple: false
target_chat:
name: Target
description: 'The chat_id to be used by the Telegram bot. !secret chat_id is not allowed on Blueprint, you will need the chat_id code. '
notification:
name: Notification
description: 'Select "true" to disable notification, leave "false" to receive notification. '
selector:
select:
options:
- 'true'
- 'false'
multiple: false
sort: false
custom_value: false
default: 'false'
prompt:
name: Gemini Prompt
description: Customise your AI prompt for the image analysis.
default: Very briefly describe what you see in this image from my cctv camera. Your message needs to be short to fit in a phone notification. Don't describe stationary objects or buildings. Only describe animals, vehicles or persons.
ai_entity:
name: AI Task Entity (Optional)
description: Select the Google/Gemini or OpenAI entity to use for the image analysis. Defaults to the system default if left blank.
default:
selector:
entity:
domain: ai_task
multiple: false
base_url:
name: (Optional) Base URL
description: The external url for your Home Assistant instance. This will default to a relative URL and will open the clips in the app instead of the browser, which may cause issues on some devices.
default: ''
zone_filter:
name: (Optional) Zone Filter
description: Only notify if object has entered a defined zone.
default: false
selector:
boolean: {}
zones:
name: (Optional) Trigger Zones
description: A list (-) of zones you wish to recieve notifications for.
default: []
selector:
object: {}
labels:
name: (Optional) Trigger Objects
description: A list (-) of objects you wish to recieve notifications for.
default: []
selector:
object: {}
presence_filter:
name: (Optional) Presence Filter
description: Only notify if selected presence entity is not "home".
default: ''
selector:
entity:
multiple: false
source_url: https://gist.github.com/jakesOneFourSeven/cc8fab1e8712a34dbd596ebd355d76ea
mode: single
max_exceeded: silent
trigger_variables:
input_camera: !input camera
camera: '{{ input_camera | replace(''camera.'', '''') }}'
trigger:
- platform: mqtt
topic: frigate/events
payload: '{{ camera }}/new'
value_template: '{{ value_json[''after''][''camera''] | lower | replace(''-'',''_'') }}/{{ value_json[''type'']}}'
id: frigate-event
variables:
id: '{{ trigger.payload_json[''after''][''id''] }}'
input_camera: !input camera
camera: '{{ input_camera | replace(''camera.'', '''') }}'
camera_name: '{{ camera | replace(''_'', '' '') | title }}'
target_chat: !input target_chat
object: '{{ trigger.payload_json[''after''][''label''] }}'
label: '{{ object | title }}'
entered_zones: '{{ trigger.payload_json[''after''][''entered_zones''] }}'
type: '{{ trigger.payload_json[''type''] }}'
base_url: !input base_url
zone_only: !input zone_filter
input_zones: !input zones
zones: '{{ input_zones | list }}'
input_labels: !input labels
labels: '{{ input_labels | list }}'
presence_entity: !input presence_filter
notification: !input notification
clip_local_file: '/config/www/clip_{{ id }}.mp4'
snapshot_file: '/config/www/snapshot_{{ id }}.jpg'
condition:
- '{{ input_camera is not none and input_camera | string | length > 0 }}'
- '{{ type != ''end'' }}'
- '{{ not zone_only or entered_zones|length > 0 }}'
- '{{ not zones|length or zones|select(''in'', entered_zones)|list|length > 0 }}'
- '{{ not labels|length or object in labels }}'
- '{{ not presence_entity or not is_state(presence_entity, ''home'') }}'
action:
- action: camera.snapshot
data:
entity_id: !input camera
filename: '{{ snapshot_file }}'
- delay: 0.5
- action: ai_task.generate_data
target:
entity_id: !input ai_entity
data:
task_name: Frigate Snapshot Analysis
instructions: !input prompt
attachments:
- selector:
file:
path: '{{ snapshot_file }}'
response_variable: generated_content
- action: telegram_bot.send_photo
data:
target: '{{ target_chat }}'
disable_notification: '{{ notification }}'
caption: '{{ generated_content.data }}'
file: '{{ snapshot_file }}'
- repeat:
sequence:
- wait_for_trigger:
- platform: mqtt
topic: frigate/events
payload: '{{ id }}'
value_template: '{{ value_json[''after''][''id''] }}'
timeout:
minutes: 2
continue_on_timeout: false
- condition: template
value_template: '{{ wait.trigger.payload_json[''type''] == ''end'' }}'
- action: downloader.download_file
data:
overwrite: true
url: '{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4'
filename: 'www/clip_{{ id }}.mp4'
- action: telegram_bot.send_video
data:
target: '{{ target_chat }}'
disable_notification: '{{ notification }}'
caption: '{{ generated_content.data }}'
file: '{{ clip_local_file }}'
until: '{{ wait.trigger.payload_json[''type''] == ''end'' }}'
@jakesOneFourSeven
Copy link
Copy Markdown
Author

jakesOneFourSeven commented Mar 30, 2024

Changes

2024-03-30

  • service: google_generative_ai_conversation.generate_content incorporated into telegram messages.
  • Added 2 second delay after downloader because the AI was sometimes using the old snapshot.jpg
  • Need to add the following to configuration.yaml:

downloader:
download_dir: /config/www/downloads

2024-03-31

  • Added input field for AI prompt in order to allow per automation prompt.
  • Added camera selector dropdown in blueprint

@guikof97
Copy link
Copy Markdown

Hello friend, I really liked it and I use it!
Is there any way to adapt this code for ollama and use it locally?

@jakesOneFourSeven
Copy link
Copy Markdown
Author

Hello friend, I really liked it and I use it! Is there any way to adapt this code for ollama and use it locally?

Hi, glad to hear you are enjoying the changes. I haven't tried using Ollama, not hearing great things at the moment.

You could replace the following action that calls the Gemini service with the equivalent service for Ollama:

"- service: google_generative_ai_conversation.generate_content
data:
prompt: '>-{{ prompt }}'
image_filename: www/downloads/snapshot.jpg
response_variable: generated_content"

I will try using Ollama once it matures a little bit more.

@jakesOneFourSeven
Copy link
Copy Markdown
Author

I've added an option to send the response from gemini to a google speaker, do you want me to update the gist to include that option?

"- action: tts.google_say
data:
cache: false
entity_id: '{{ speaker_target }}'
message: ' {{ generated_content.text }} ' "

@guikof97
Copy link
Copy Markdown

Hello friend, I really liked it and I use it! Is there any way to adapt this code for ollama and use it locally?

Hi, glad to hear you are enjoying the changes. I haven't tried using Ollama, not hearing great things at the moment.

You could replace the following action that calls the Gemini service with the equivalent service for Ollama:

"- service: google_generative_ai_conversation.generate_content data: prompt: '>-{{ prompt }}' image_filename: www/downloads/snapshot.jpg response_variable: generated_content"

I will try using Ollama once it matures a little bit more.

I understand, my friend, I'll wait too!
Even though Gemini is "free", I don't feel comfortable with so much information being given to Google.

I've added an option to send the response from gemini to a google speaker, do you want me to update the gist to include that option?

"- action: tts.google_say data: cache: false entity_id: '{{ speaker_target }}' message: ' {{ generated_content.text }} ' "

For me, it wouldn't work, because there are 8 cameras and 2 are pointed at the street. I would never stop talking lol.
I would really like to get answers in my primary language, pt-BR. I tried to modify the prompt and even asking in pt-BR or asking to answer in pt-BR, it keeps answering in en-US.

@guikof97
Copy link
Copy Markdown

guikof97 commented Nov 10, 2024

I created an automation to alert cameras that were offline for more than 1 minute and report again which ones are still online. The interval for resending messages is every 5 minutes. If there are no more, it reports "All cameras are online". I tried using the "initial_delay" variable in the loop but it always gave an error... :S with this exception, could you check and give your opinion @jakesOneFourSeven ?
Code:

Automation_Home_Assistant_Telegram_Notify_and_Check_Offline_Cameras_Frigate

@mrfixitpa
Copy link
Copy Markdown

This doesn't work for me. Error while running automation.
The automation "Living_Room_Camera_Notify_with_AI" (automation.living_room_camera_notify_with_ai) has an unknown action: downloader.download_file.

This error prevents the automation from running correctly. Maybe this action is no longer available, or perhaps a typo caused it.

Executed: November 11, 2024 at 7:27:44 PM
Error: Action downloader.download_file not found
Result:
params:
domain: downloader
service: download_file
service_data:
overwrite: true
url: >-
https://my-web-address/api/frigate/notifications/1731371263.671046-ohz2no/snapshot.jpg
filename: snapshot.jpg
target: {}
running_script: false

@guikof97
Copy link
Copy Markdown

Hey @mrfixitpa , look and adjust your config of HA to add:

downloader: download_dir: /config/www/downloads

View first comment :)

Changes

2024-03-30

  • service: google_generative_ai_conversation.generate_content incorporated into telegram messages.

  • Added 2 second delay after downloader because the AI was sometimes using the old snapshot.jpg

  • Need to add the following to configuration.yaml:

downloader: download_dir: /config/www/downloads

2024-03-31

  • Added input field for AI prompt in order to allow per automation prompt.
  • Added camera selector dropdown in blueprint

@mrfixitpa
Copy link
Copy Markdown

Thanks for this. Apparently that is also not the fix. I did what your reply suggested and I received and immediate error. It seems now the downloader is now within the integrations. I have made the changes and no errors. Hope this works :)

@guikof97
Copy link
Copy Markdown

I had the same problem as you, I just didn't remember how I solved it, if I'm not mistaken, just download the service in settings/Devices and services - "Downloader" and in the configuration you enter the path. I checked my .yaml and I don't have anything from the first comment. Remember configure the Google Gemini for HA.

@RGR66
Copy link
Copy Markdown

RGR66 commented May 9, 2025

Im having an issue with the blueprint. It doesn't seem to be saving the snapshot to the right place, and when I try to search for it to send it via Telegram, it generates a 404 error.
This is shown in the HA log. Can you help me?

05-25-09 12:36:56.306 WARNING (MainThread) [homeassistant.components.telegram_bot] Status code 404 (retry #1) loading http://192.168.11.206:8123/api/frigate/notifications/1746787003.621033-jq87ul/snapshot.jpg
05-25-09 12:36:57.354 WARNING (MainThread) [homeassistant.components.telegram_bot] Status code 404 (retry #2) loading http://192.168.11.206:8123/api/frigate/notifications/1746787003.621033-jq87ul/snapshot.jpg
2025-05-09 12:36:58.409 WARNING (MainThread) [homeassistant.components.telegram_bot] Status code 404 (retry #3) loading http://192.168.11.206:8123/api/frigate/notifications/1746787003.621033-jq87ul/snapshot.jpg
2025-05-09 12:36:59.456 WARNING (MainThread) [homeassistant.components.telegram_bot] Status code 404 (retry #4) loading http://192.168.11.206:8123/api/frigate/notifications/1746787003.621033-jq87ul/snapshot.jpg
2025-05-09 12:37:00.471 WARNING (MainThread) [homeassistant.components.telegram_bot] Status code 404 (retry #5) loading http://192.168.11.206:8123/api/frigate/notifications/1746787003.621033-jq87ul/snapshot.jpg
2025-05-09 12:37:00.471 WARNING (MainThread) [homeassistant.components.telegram_bot] Can't load data in http://192.168.11.206:8123/api/frigate/notifications/1746787003.621033-jq87ul/snapshot.jpg after 5 retries

@jakesOneFourSeven
Copy link
Copy Markdown
Author

jakesOneFourSeven commented Nov 24, 2025

Had to update the code due to changes in how Gemini is used in HA. Re-import the blueprint and add the following to Configuration.yaml

homeassistant:
media_dirs:
# Alias name: Full absolute path to the directory
frigate_snaps: /config/www

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment