Microsoft Certified: Machine Learning Operations Engineer Associate
Independent: not affiliated with Microsoft. Free to try; the study bundle adds a study guide and two practice exams.
5 original questions, none of them in the practice exams.
Pick an answer. Show answer explains every option and links its Microsoft source.
5 questions / Explanations link Microsoft sources / Last updated Oct 26, 2026
Question 1
Multiple response
Domain 1 / Task 1.3 Implement IaC for Machine Learning
A regulated team must be able to tie every model registered in an Azure Machine Learning workspace to the commit of the training code that produced it. Data scientists submit command jobs with az ml job create from clones of the team's GitLab repository on their own laptops, and the team does not want anyone adding tags by hand. Which two conditions must hold for Azure Machine Learning to record the repository URI, branch and commit hash as properties of each job? Choose two.
Show answerHide answer
Answers: A and D
Why
When a job is submitted from the SDK or the CLI, the upload step checks whether the source files sit in a Git repository, and if the git command is available it records the repository URI, branch, commit hash and a dirty flag as job properties such as mlflow.source.git.commit and azureml.git.commit. If git is missing on the submitting machine, or the training files are not inside a repository, nothing Git-related is tracked.
Why not the others
BTracking happens on whichever machine submits the job; a compute instance is one place to work with Git, not a requirement.
CThe information comes from the local clone, so the remote can be any Git-compatible service, GitLab and Bitbucket included.
EA dirty working tree is still tracked: the azureml.git.dirty property is set to True, which shows that the job ran uncommitted changes, and nothing has to be pushed.
Domain 2 / Task 2.4 Monitor and maintain models in production
A churn model in MLflow format has been scoring live requests on a managed online endpoint in Azure Machine Learning for several weeks, and the endpoint's requests-per-minute metric is steady. The team uses az ml schedule to create an out-of-box model monitor for the deployment, with the default data drift, prediction drift and data quality signals. Every scheduled run ends with no production data to analyze, and the workspace holds no data assets for the deployment's model inputs or outputs. What should the team do?
Show answerHide answer
Answer: A
Why
The monitor analyzes the production inference data that data collection logs to Blob Storage and registers as data assets named after the endpoint and deployment, and Microsoft's model monitoring page lists data collection enabled on the deployment as a prerequisite for monitoring an online endpoint. Missing model-input and model-output assets while the endpoint serves steady traffic mean that nothing is being logged. Data collection can be enabled on an existing deployment, and for an MLflow model, turning it on makes Azure Machine Learning log the inputs and outputs without changes to a scoring script.
Why not the others
BThe look-back window only selects which slice of the production data asset a run reads, and no model-input or model-output asset exists for any window to cover.
CA reference data asset is optional, because the monitor compares against recent past production data by default; training data would replace only the comparison side and still leave no production data to analyze.
DApplication Insights integration sends the endpoint's built-in metrics and logs for analysis in Application Insights, while the monitor reads the production inference data asset that data collection registers.
Domain 3 / Task 3.2 Deploy and manage foundation models
A support assistant calls a Standard deployment of an Azure OpenAI model in a Microsoft Foundry resource. After a weekend with no changes from the team, the deployment reports a different model version from the one that was evaluated and the answers changed format. The team wants to stay on the version it tested until it chooses to move, but it does not want the deployment to stop working when that version is retired. Which versionUpgradeOption should the deployment have?
Show answerHide answer
Answer: B
Why
With OnceCurrentVersionExpired the deployment keeps its selected version until that version's retirement date, when it automatically upgrades to the current default version. The unplanned change is the behavior of OnceNewDefaultVersionAvailable, which upgrades within two weeks of a new version being designated the default. Automatic updates apply only to Standard deployment types, and the option can be changed in the Foundry portal, through REST or with Azure PowerShell; the Azure CLI can read it but not update it.
Why not the others
AThat is the setting behind the surprise: the deployment upgrades within two weeks of a new version being designated the default.
CNoAutoUpgrade never upgrades, so at the retirement date the deployment stops working until the code points at a different deployment.
DAutomatic updates apply only to Standard deployment types, so a provisioned deployment never moves off a retiring version on its own; the team would have to migrate it by hand, in place or to a new deployment, before the retirement date, while the Standard deployment needs only its option changed.
Domain 4 / Task 4.1 Evaluation and validation for apps and agents
A claims assistant in a Microsoft Foundry project must quote the policy number in every answer as POL- followed by eight digits. The team runs batch evaluations before each release and wants a per-item score for this rule that is deterministic, calls no judge model and runs in the Foundry evaluation service rather than on infrastructure the team hosts. Which custom evaluator should they add to the project's evaluator catalog?
Show answerHide answer
Answer: C
Why
A code-based evaluator is a Python grade() function that scores each item with deterministic logic and returns a float from 0.0 to 1.0; Microsoft's custom evaluators page lists rule-based checks, keyword matching and format validation as what it is best for. It runs in a sandboxed Python environment with no network access and calls no LLM, although deployment_name and pass_threshold still have to be passed as initialization parameters.
Why not the others
AAn endpoint-based evaluator delegates scoring to an external HTTP endpoint that the team owns and operates, outside the evaluation service.
BThe custom evaluators page names output format compliance as an example of criteria that go beyond the built-in catalog and call for a custom evaluator.
DA prompt-based evaluator uses a judge prompt that an LLM evaluates for each item, which the requirement rules out; the custom evaluators page lists it as best for subjective judgments such as tone.
Domain 5 / Task 5.2 Advanced fine-tuning and model customization
A team uses reinforcement fine-tuning (RFT) in Microsoft Foundry to teach a model to write SQL for questions about a sales database. Their grader gives full credit when a generated query references the same tables and columns as the reference query. Mean reward climbs steadily on both the training and the validation questions, but on a held-out test set from the same database, where each query is run and its result compared with the expected rows, accuracy is flat at every checkpoint. What is the most likely cause, and what should the team do first?
Show answerHide answer
Answer: D
Why
Microsoft's fine-tuning guidance lists misleading reward signals as an RFT challenge: a rising reward does not establish better task performance if the model exploits the grader, and the remedy is to test graders against correct, incorrect and adversarial responses. A grader that looks only at which tables and columns a query names can be satisfied by queries that return the wrong rows, so reward can rise on the training and validation questions while the held-out test, which scores actual results, stays flat.
Why not the others
AComparing checkpoints on held-out tasks is sound practice, but overfitting would show validation reward falling behind training reward, and here both climb while accuracy is flat at every checkpoint, so there is no better checkpoint to keep.
BValidation reward climbs too, which shows the model already names the right tables and columns for questions it was not trained on, so coverage is not what is missing, and more questions scored by the same grader would reinforce the same shortcut.
CRFT fits tasks where several solutions are possible and their quality can be scored, as with SQL queries, while DPO needs preferred and rejected responses to the same input; the shortfall lies in what this grader rewards, not in the method.
The study bundle has two practice exams with 120 original questions, an answer key and an explanation for every answer, plus the 59-page study guide. Practice exam explanations are shorter than the ones on this page: a short paragraph per question, with no source links.
Study bundle
$19
120 practice questions / Study guide / Answer keys
Every bundle includes 6 months of access: download the PDFs, and any new editions, until 6 months after your purchase date. Your receipt has a license key. Enter it at overqual.com/download for the three PDFs, stamped with your email and order number. They are yours to keep.