By “safely test” we mean “not accidentally spamming anyone with test incidents”.
Automated tests are well and good, but filters especially can be complex thingsand can behave in what for users is an unexpected way. (The problem sometimesdoes exist between keyboard and chair.)
The techniques here are suitable both for problems in production, new types offilters, unexpected behaviours, refactors of the entire shebang, and workingout how things actually work in order to improve automated tests.
Testing filters, profiles, timeslots regardless of destination type¶
When doing a test of everything except details of the destination type, usethe standard email medium(argus.notificationprofile.media.email.EmailNotification
) as the soledestination type. Adapt or create a profile accordingly.
Automated tests¶
Use the override_settings decorator to override the settingMEDIA_PLUGINS
with the exact list of media the test will need.
Redirect email to console¶
Do not actually send those notification emails, catch them locally.
There are two ways of doing this:
Changing the EMAIL_BACKEND setting to
django.core.mail.backends.console.EmailBackend
. Sending an email willthen turn up in some terminal window somewhere.Sending to an email server you control, the easiest is to send it to oneset up for the test. Set
EMAIL_HOST
tolocalhost
,EMAIL_PORT
to1025
, and run a dummy mailserver:See Also2023 Conference – NeurIPS Blog$ python -m smtpd -n -c DebuggingServer localhost:1025
Sent notifications will then be dumped to the console where the dummy serverruns.
The former method works less well with docker since it can be tricky to findthe correct container that has the dump of the email, plus there’s usually somuch going on in the console of the container that finding the message might betricky. With the latter method the email is dumped in the window theDebuggingServer is running in, which allows a little more control.
One of the two methods above is probably what you want for yourdevelopment-setup regardless.
Automated tests¶
Use the override_settings decorator to override the settingEMAIL_BACKEND
and use method 1 above.
Testing something that fails in production¶
Copy the profiles to your debugging-setup. You can just reuse yourdevelopment-setup for this: move the dev-database out of the way (for instancewith the ALTER DATABASE .. RENAME
statement), dump the production database,then import the dump into your dev-database.
Deactivate profiles you are not testing¶
Deactivating profiles prevents spam, makes the test go faster since there isless to test, and makes it easier to find the result in the console since lesswill be printed. (Don’t do this in production regardless.)
Find the ids of the profiles you want to test first. Let’s assume you’ll betesting profiles “1” and “22”.
There are several methods:
Deactivate all others in the admin.
If you have access to the database, deactivate via
python manage.py dbshell
:argus=> UPDATE argus_notificationprofile_notificationprofile SET active = false WHERE id NOT IN (1, 22);
You can deactivate via the python shell as well, launch with
pythonmanage.py shell
:> from argus.notificationprofile.models import NotificationProfile> NotificationProfile.objects.exclude(id__in=(1, 22)).update(active=False)
With option 1 you need an authenticated user in the system that has bothis_staff
and is_superuser
set to True
. For all the others you needdatabase access.
Automated tests¶
Use factories to create exactly what to test. You can recreate a scenario inthe production database exactly with the factories, then when the test works,adjust the factory to only set exactly the subset of attributes needed.
Testing specific destination types¶
If the plugin has a dummy version that just dumps to the console, see iftesting with that dummy is sufficient. Otherwise you need to find a testdestination (in order to not spam the production destination). For instancewith the MS teams-plugin set it up to spam a test-channel or yourself.
If the live test works but pushing to the production destination does not, it’sprobably a permissions-problem.
Email is not delivered¶
If an email is not delivered even if the email server logs of the first hop(the one set as EMAIL_HOST
) says the email was received, it is probably oneof three things:
Something is wrong with the first hop, like a missing or outdatedcertificate, wrong entry in DNS, missing DKIM/SPF/DMARC, missing reverse DNSlookup, the IP address is on a blacklist. The possibilities are nearlyendless.
Something is wrong at the final hop (the email server receiving for thedomain in the
To:
-address)Something is wrong in between. Network, DNS, other email servers along theroute…
This is not something you can solve alone. Send a fake incident to the failingemail address with the normal email backend in settings and use that to findthe exact:
date and time the non-delivered email was sent (do not forget the timezone)
the
To:
-addressthe
From:
-address
Give this to your postmaster (the person or persons responsible for/monitoringthe email system). That should be enough for them to sometimes solve case #1.
Making test incidents to trigger the notification system¶
While it is possible to sit in the python shell and create everything by handfor 100% control, most of the time it is sufficient to use one of the belowmethods:
In the admin, go to the
Incidents
model in the “ARGUS_INCIDENT” section.There’s a button “FAKE INCIDENT” just left of the “ADD INCIDENT”-button.From the command line use the
create_fake_incident
command withpythonmanage.py
.
We do not recommend making an incident directly in the database, because theevent that triggers the notification is made by Python.