Migrating from Microsoft Monitoring Agent (MMA) to Azure Monitor Agent (AMA)

Discussions points:

Why upgrade / modernize to the Azure Monitor Agent?

As per the Microsoft Azure Update announcement released on the 19th August 2021, Microsoft will be retiring the Log Analytics Agent (aka OMS / MMA)  on 31 August 2024 which means that you will need to have migrated and started using the new Azure Monitor Agent (AMA) to monitor your virtual machines before this deprecation date.

The Azure Monitor Agent has been created as a “single agent” to consolidate legacy agents such as the Log Analytics Agents (also referred to as “MMA” or “OMS” agent) within the Azure Monitor as a single pane of glass.

The Azure Monitor Agent upgrade applies to both your Windows and Linux virtual machines residing in both, your Azure cloud and your hybrid environments such as any on-premises servers and other 3rd party cloud providers virtual machines.

For all virtual machines outside of Azure cloud, the Azure Arc agent is required – which incurs no cost when used only for AMA connectivity.

How can Azure Monitor Agent be deployed?

The Azure Monitor Agent offers a simplified one-touch type deployment methodology versus the MMA agent deployment which required the agent installation, plus a workspace ID and Primary Key.

You may deploy the Azure Monitor Agent based on your bespoke environment and requirements:

#1 If you need to remain cognizant of subscription based boundaries you may choose to deploy the Azure Monitor Agent using the Microsoft for Defender (MDC) which will automatically deploy the AMA on all existing and newly deployed virtual machines in the respective subscription.

#2 However if you are not restricted by subscription boundaries, you can instead choose to provision the Azure Monitor Agent to multiple virtual machines across multiple subscriptions by using either the Azure Policy or Azure Monitor based Data Collection Agent (DCR) tool.

How much does the Azure Monitor Agent cost?

As part of the Continual Cloud Cost Optimization, it is vital to monitor, manage and minimize our clients cloud expenditure.

#1 The Azure Monitor Agent itself incurs no cost, however remain cognizant of the inter-dependent components below,

#2 You might incur storage charges for the ingested data-at-rest inside the Log Analytics Workspace destination endpoint.

For information on Log Analytics data collection and retention and for customer metrics, follow this link: https://azure.microsoft.com/pricing/details/monitor/

#3 Another important cost factor that I have noticed is that a few customers overlook the incurred bandwidth egress costs between the source virtual machines and the destination Log Analytics Workspaces. Keep in mind that if your source virtual machines and destination Log Analytics Workspace (LAW) is in another region, then you will incur intra-region / inter-region egress bandwidth costs. For more information, the pricing can be found here https://azure.microsoft.com/en-us/pricing/details/bandwidth/

What are the benefits of migrating to AMA?

Besides the obvious 31 August 2024 deprecation deadline of the Log Analytics Agent (MMA), there are additional benefits to migrating / modernizing to the newer AMA agent.

Azure Monitor Agent provides  benefits for cost savings, simplified management experience, security and performance.

The Azure Monitor Agent (AMA) provides the following benefits over legacy MMA agents:

#1 Security and performance

• Enhanced security is provided through the Managed Identity management and Azure Active Directory (Azure AD) tokens (for clients).
• AMA offers a higher events-per-second (EPS) upload rate in comparison to MMA.

#2 Cost savings:

Using DCRs is one of the most useful advantages of using Azure Monitor Agent  (AMA):

• DCRs offers you the opportunity to configure both, default and customized data collection using xpath filtering for specific virtual machines connected to a single or multiple (multihomed) workspaces. As compared to the “all or nothing, non filtered” approach of the MMA legacy agents. Using AMA means reduced data ingestion volumes by filtering out unwanted metrics / logs resulting in lower ingested data and reduced traffic costs.
• With DCRs, you can define which data to ingest and which data to filter out to reduce workspace clutter based on xpath filtering and to save on storage costs.

#3 Simpler management of data collection, including ease of troubleshooting:

• Easy multihoming on Windows and Linux by having the option of linking multiple DCRs to a single virtual machine and connecting the single source vm to multiple destination Log Analytic Workspaces.
• Centralized, agent configuration makes every action simpler and more easily scalable throughout the data collection lifecycle, from onboarding to deployment to updates and changes over time.
• Greater transparency and control of more capabilities and services, such as Microsoft Sentinel, Defender for Cloud, and VM Insights.

#4 A single agent approach that consolidates all features necessary to address all telemetry data collection needs across servers and client devices running Windows 10 or 11. A single agent is the goal, although Azure Monitor Agent can converge with the Log Analytics agents when / if required.

Which Deployment Options are available to me?

Before deciding on your Azure Monitor Agent deployment methodology, identify the end state of your Azure Monitor Agent infrastructure. Do you want each of the subscriptions to send their Azure Monitor Agent metrics to subscription specific Log Analytics Workspaces or to a single Log Analytics Workspace for centralized tenant-level monitoring?

#1 If you need to remain cognizant of subscription based boundaries while deploying the AMA agent then using the Microsoft Defender for Cloud (MDC) may be a great option for you. The automated deployment of the AMA at a subscription level will be configured via the Defender for Cloud console whereby the AMA will be configured and pointed to a Log Analytics Workspace in the local subscription. This ensures that all existing and newly deployed vms in the target subscription boundary are automatically provisioned with the AMA agent and automatically joined to the subscription specific destination Log Analytics Agent to ensure that all vms are monitored across the subscription. This however means that if you have multiple subscriptions in your tenant that you will have a LAW per subscription in your tenant.

To enable the automated AMA agent provisioning per subscription using MDC go to:

Microsoft Defender for Cloud (MDC) >

Environment Settings > select target subscription >

Click on Server Defender Plan > Monitoring Coverage > click on Settings

(Make sure the Defedner Plan for Servers is ON)

Under the Servers Defender Plan > Log Analytics agent/Azure Monitor agent > Configuration > select Edit Configuration

Select Azure Monitor Agent radio button and then select the custom destination Log Analytics Workspace (unless you are going to use the default workspace) > Apply

#2 Microsoft also offers the option of using Azure Policy to onboard a large number of virtual machines at scale, as well as the automatic provisioning of newly deployed virtual machines.

#3 If you have multiple subscriptions but are not restricted by subscription boundaries and would prefer not having multiple AMA Log Analytics Workspace collection endpoints per subscription boundary and would instead prefer to have a single centralized Log Analytics Workspace endpoint for all the virtual machines in the tenant, then choosing to use an Azure Monitor DCR based methodology would be a better fit for purpose. This methodology, unlike the Azure Policy, requires manual updating of the source virtual machines in the Data Collection Rule (DCR).

What is a Data Collection Rule (DCR)?

What are the components of the DCR?

How to setup a DCR for AMA?
https://learn.microsoft.com/en-za/azure/azure-monitor/agents/data-collection-rule-azure-monitor-agent?tabs=portal

Data resiliency and high availability:

When a DCR rule gets created and stored in a particular region it is backed up to the paired-region within the same geography. The service is deployed to all three availability zones within the region. For this reason, the Azure Monitor DCR is a zone-redundant service, which further increases availability.

Microsoft’s Migration guidance:

Phase 1 – Pre-preparation / Due Diligence:

#1 As stated above, for all virtual machines outside of Azure cloud, the Azure Arc agent installation is required which incurs no cost. (If you dont have any hybrid servers, then this doesn’t apply to you, then skip this step).

#2 Do you have legacy Azure solutions eg OMS that rely on the MMA agent to collect specific data? (If this doesn’t apply to you, then skip this step), else:

2.1 If you are unsure of whether there are any legacy solutions in place, then use the AMA Migration Helper to discover solutions enabled on your workspaces that use the legacy agents, including the per-solution migration recommendation.

Start testing for these solutions during the preview phase. This will save you time, avoid surprises later and ensure you are ready to deploy to production as soon as the service becomes generally available.

The AMA Migration Helper is a workbook-based Azure Monitor solution that helps you discover what to migrate and to track the progress as you move from the Log Analytics Agent to the Azure Monitor Agent. Use this single pane of glass view to expedite and track the status of your agent migration journey. 

The AMA Migration Helper walkthru can be found here:

From <https://learn.microsoft.com/en-za/azure/azure-monitor/agents/azure-monitor-agent-migration-tools#using-ama-migration-helper>

If you plan to integrate with Microsoft Sentinel, then use the  Gap analysis for Microsoft Sentinel for a comparison of the additional beneficial data that can be collected by Microsoft Sentinel after having migrated onto the AMA agent

#3 Agent coexistence:

• If you are setting up a Greenfields environment with resources, using Infrastructure-as-Code such as deployment scripts and onboarding templates, Microsoft recommends installing Azure Monitor Agent (AMA) together with a legacy MMA agent in your new environment to decrease the migration effort later.
• Azure Monitor Agent can run alongside the legacy Log Analytics agents on the same virtual machine so that you can continue to use existing functionality during evaluation or migration. Before you begin the transition, make sure that you understand the limitations below:
  • Be aware when you collect duplicate data from the same machine, as this could skew query results and affect downstream features like alerts, dashboards, workbooks and generate more charges for the dual agent data ingestion and dual agent data retention.

To avoid data duplication, ensure that the agents are either:

• collecting data from different machines or 
• If using the same virtual machine source then make sure to send the data to different Log Analytics Workspace destinations.

• For Defender for Cloud, you will only be billed once per machine when running both MMA and AMA agents
• For Sentinel, you can easily disable the legacy connector to stop ingestion of logs from legacy MMA agents.
• Running two telemetry agents on a virtual machine consumes double the vm resources, including but not limited to CPU, memory, storage space, and network bandwidth.

From <https://learn.microsoft.com/en-za/azure/azure-monitor/agents/azure-monitor-agent-migration>

Migration Steps

1.  Create data collection rules. You can use the DCR generator to automatically convert your legacy agent configuration into data collection rule templates. Review the generated rules before you create them, to leverage benefits like filtering, granular targeting (per machine), and other optimizations.

Use the DCR Config Generator tool to parse Log Analytics Agent configuration from your workspaces and generate/deploy corresponding data collection rules automatically. You can then associate the rules to machines running the new agent using built-in association policies.

The DCR generator currently only converts the configurations for Windows event logs, Linux syslog and performance counters. Support for additional features and solutions will be available in the future.

From <https://learn.microsoft.com/en-za/azure/azure-monitor/agents/azure-monitor-agent-migration>

2. Deploy extensions and DCR-associations:

2.1 Test first by deploying extensions and DCR-Associations on a few non-production machines. You can also deploy side-by-side on machines running legacy agents (see the section above for agent coexistence. The additional extension list can be found here https://learn.microsoft.com/en-za/azure/azure-monitor/agents/agents-overview#supported-services-and-features

2.2 Once data starts flowing via Azure Monitor agent, compare it with legacy agent data to ensure there are no gaps. You can do this by joining with the Category column in the Heartbeat table which indicates ‘Azure Monitor Agent’ for the new data collection

2.3 Post testing, you can (optionally, but not required) roll out broadly using Azure Policy built-in policies for at-scale deployment of extensions and DCR-associations. Using Azure Policy will also ensure automatic deployment of extensions and DCR-associations for any new machines in future.

Before you deploy a large number of AMA agents, consider configuring the workspace (alternate destination LAW for MMA agents) (to be disabled at a later stage) data collection for the Log Analytics agent. If you leave data collection for the Log Analytics agent enabled, you might collect duplicate data and increase your costs. You might choose to collect duplicate data for a short period during migration until you verify that you’ve deployed and configured Azure Monitor Agent correctly.

2.4 Use the AMA Migration Helper to monitor the at-scale migration across your machines

3. Validate that Azure Monitor Agent is collecting data as expected and all downstream dependencies, such as dashboards, alerts, and workbooks, function properly.

4. Clean up: After you confirm that Azure Monitor Agent is collecting data properly, you may choose to either disable or uninstall the legacy Log Analytics agents to reduce cloud expenditure and bill shock.

5. If you have migrated to Azure Monitor Agent for selected features/solutions and you need to continue using the legacy Log Analytics for other legacy / bespoke solutions, this is an option available to you.

6. If you’ve migrated to Azure Monitor agent for all your requirements, you may uninstall the Log Analytics agent from monitored resources. Clean up any configuration files, workspace keys, or certificates that were used previously by the Log Analytics agent.

7. Do not uninstall the legacy agent if you need to use it for uploading data to System Center Operations Manager.

Cost Investigation

As part of Cost Management in the cloud, you may want to know what your Azure Monitor costs are. Below are 2 simple ways to quickly analyse your Azure Monitor charges on your Azure bill as well as how to estimate your Azure Monitor costs.

Azure Monitor uses a consumption-based pricing model whereby you only pay for what you use. Default Azure Monitor features don’t incur any charges. You do however pay for the ingestion and retention of data that is collected in your Log Analytics Workspace. The following table describes the different types of usage that are charged in Azure Monitor.

Detailed pricing can be found here.

The bulk of your costs typically come from data ingestion and retention for your Log Analytics workspaces and Application Insights resources. It is difficult to accurately estimate your data volumes because consumption will vary based on your bespoke configuration.

Below are 2 methods you can use, namely:

1. Azure Calculator for estimation
2. Current Azure Monitor usage and charges
   • Option #1 Using Cost Management + Billing
   • Option #2 Using Log Analytics Logs instead of Cost Management + Billing

1. Azure Calculator Estimation

1. Azure Calculator

You can use the Azure calculator to estimate your Azure costs, make sure to select you region. Click here.

2. View your current Azure Monitor usage and charges

Keep in mind that to view the cost analysis for your scope, you will need the appropriate permissions, which are explained here

Option #1 Using Cost Management + Billing

Portal > Cost Management + Billing >  Cost Management > Cost analysis > Select your subscription or scope, time period and create a filter based on Meter Category = Log Analytics. (Try adding a filter for Meter Category = Azure Monitor depending on your Tier)

*Depending on what tier your LAW is using, will determine your filter – usage from your Log Analytics workspaces can be found by first filtering on the Meter Category column to show Log Analytics, Insight and Analytics (used by some of the legacy pricing tiers), and Azure Monitor (used by commitment-tier pricing tiers).

You can search for any of these filters, based on your bespoke environment:

Usage for Azure Monitor Logs (Log Analytics) can be billed with the Log Analytics service (for Pay-as-you-go Data Ingestion and Data Retention).

Other services such as Microsoft Defender for Cloud and Microsoft Sentinel also bill their usage against Log Analytics workspace resources, so you might want to add them to your filter.

Option #2 Using Log Analytics Logs instead of Cost Management + Billing

Alternatively, you can go to the Log Analytics workspace> Usage and Costs > open the Log query > Run.

Microsoft reference