I use Azure Sentinel and the way that Sentinel is able to analyze logs is through a Log Analytics Workspace. Some third-party platforms allow log streaming to Event Grid. So I needed a way to get my logs from Event Grid into the Log Analytics workspace.
You might think this would be a simple task…moving data from one Azure resource to another. But it turns out there is no native way to do this. Additionally, I was not able to find much documentation or any guides for this process either, so here I am writing this up for anyone who cares.
I based this flow largely on the GitHub Scan Audit connector that is available via the Sentinel Content Hub.
In a nutshell, the process is to create an Event Subscription on the Event Grid that will trigger a Function App that will deliver logs to the Log Analytics Workspace API.
Third-Party Platform --> Event Grid --> Function App --> Log Analytics Workspace
This guide does not cover actually getting logs from a third-party platform to Event Grid. In most cases, the third-party vendor will have documentation for this process.
Your permission model or other proclivities of your Azure environment may make some of these steps look a little different.
Log Analytics Workspace and Sentinel
If you do not already have these resources deployed, do so. The deployment of those resources is beyond the scope of this document.
Under Settings > Tables, create a new table (MMA-Based). This will require a sample log output. Hopefully you can get this via the third-part platform's documentation but you may also have to mock up an API call to get the log format. Save the format as a file and upload. Set the appropriate delimiter; in most cases "new line" will suffice. For the collection path we can use a dummy path, so select Linux and set the path to
/tmp. Set an appropriate log name and create.
To get the log sample, as an alternative method, you can create the Function as described below and utilize the default code that is generated to print out logs. You can then view this output via the Portal in the Function's Monitor tab.
First you need to create a function app, this is the top lever resource that will hold the actual function that will be shipping the logs. This can be done following the process below in the Function section via the Microsoft Guide or done via the Portal.
Then, navigate to your Sentinel Log Analytics Workspace and then to the Agents menu under Settings. Click the "Log Analytics agent instructions" dropdown and note the
Workspace ID and
Within the Function App that you created, under Settings > Configuration, create two new Application Settings
WorkspaceKey and input the respective values noted previously.
To deploy the actual function, utilize this guide to set up a development environment in Visual Studio Code. While this may seem onerous, this was the only way that I found that I was able to meet all the necessary requirements for deployment.
As a note for the above, be sure to pay attention to what panel you are in on the side panel in VSCode. Edit in the Editor panel, deploy in the Azure panel.
In the editor, you will need to do two things.
- Create a
requirements.txtfile in the root of the folder
- Add the parsing code in
__init__.py. Replace the
sentinel_log_typevariable with your table name from the LAW minus the
Both sets of code are below.
Once the code is updated, deploy the Function via the steps in the linked guide.
Configure Event Grid Subscription
With the Function deployed, we now need to get events to the Function. Navigate to the Event Grid that is receiving logs and then to Entities > Event Subscriptions. Create a new event subscription and select Azure Function as the Endpoint Type. Click the "Select an endpoint" link and drill down to your newly created function. Create.
At this stage, all the necessary plumbing should be in place to ship your logs as desired.
Navigate to your Function App > Functions > Your Function > Monitor to see execution status and any emitted errors. Perform normal troubleshooting on your code and redeploy as needed based on the process above. Be sure to have a way to trigger events in your third-party platform if they are not frequent. Once you have successful execution, then confirm your logs are available in your Log Analytics Workspace.