Skip to content

cianfhoghlaim/cianchosaint / .agents/skills/mlflow

mlflow

Verified

references/mlflow-model-registry-deployment-reference.md

View on GitHub
# MLflow Model Registry and Deployment Reference

## Comprehensive Guide for LLM Model Management and Deployment

This document provides detailed reference documentation for MLflow's model registry, model formats, LLM-specific flavors, deployment patterns, and governance capabilities.

---

## Table of Contents

1. [Model Registry](#1-model-registry)
2. [MLflow Models Format](#2-mlflow-models-format)
3. [LLM-Specific Model Flavors](#3-llm-specific-model-flavors)
4. [Model Serving and Deployment](#4-model-serving-and-deployment)
5. [Model Governance](#5-model-governance)

---

## 1. Model Registry

The MLflow Model Registry is a centralized model store, set of APIs, and UI designed to collaboratively manage the full lifecycle of MLflow Models.

### 1.1 Core Concepts

#### Registered Model
A model stored in the registry with:
- **Unique name**: Identifier for the model
- **Versions**: Multiple versions of the same model
- **Aliases**: Mutable named references to specific versions
- **Tags**: Key-value pairs for categorization
- **Annotations**: Markdown descriptions for documentation

#### Model Version
Each registered model supports multiple versions:
- First model added is version 1
- Each subsequent registration increments the version number
- Versions track creation timestamps, source run IDs, and status

#### Model URI
Reference format for accessing models:
```
models:/<model-name>/<model-version>
models:/<model-name>@<alias>
```

### 1.2 Registering Models

#### Method 1: During Logging
```python
import mlflow.sklearn

mlflow.sklearn.log_model(
    sk_model,
    "model",
    registered_model_name="MyModel"
)
```

#### Method 2: After Experiment
```python
result = mlflow.register_model(
    model_uri="runs:/<run-id>/model",
    name="MyModel"
)
```

#### Method 3: Create Empty Model First
```python
from mlflow import MlflowClient

client = MlflowClient()
client.create_registered_model("MyModel")
client.create_model_version(
    name="MyModel",
    source="runs:/<run-id>/model",
    run_id="<run-id>"
)
```

#### UI Registration
1. Navigate to run's artifacts
2. Select the model folder
3. Click "Register Model"
4. Choose existing model or create new

### 1.3 Model Versions and Stages (Deprecated)

**Important**: Starting in MLflow 2.9, model registry stages are deprecated in favor of aliases and tags.

#### Legacy Stages (Deprecated)
- `None` - Initial state
- `Staging` - Testing/validation
- `Production` - Live deployment
- `Archived` - Deprecated models

### 1.4 Model Aliases (Recommended)

Aliases provide flexible named references for model versions, replacing the rigid stage system.

#### Setting Aliases
```python
from mlflow import MlflowClient

client = MlflowClient()

# Set alias
client.set_registered_model_alias(
    name="MyModel",
    alias="champion",
    version=1
)

# Reassign to new version
client.set_registered_model_alias(
    name="MyModel",
    alias="champion",
    version=2
)
```

#### Loading by Alias
```python
import mlflow.pyfunc

model = mlflow.pyfunc.load_model("models:/MyModel@champion")
```

#### Benefits Over Stages
- Multiple aliases per model version (enables A/B testing)
- Flexible naming (e.g., `champion`, `challenger`, `baseline`)
- Easy rollback via alias reassignment
- No fixed progression path required

### 1.5 Model Version Tags

Tags annotate model versions with metadata for tracking status and organization.

#### Setting Tags
```python
client.set_model_version_tag(
    name="MyModel",
    version="1",
    key="validation_status",
    value="pending"
)

# Update after validation
client.set_model_version_tag(
    name="MyModel",
    version="1",
    key="validation_status",
    value="passed"
)
```

#### Common Tag Patterns
```python
# Task identification
client.set_model_version_tag("MyModel", "1", "task", "text-classification")

# Environment tracking
client.set_model_version_tag("MyModel", "1", "environment", "production")

# Approval status
client.set_model_version_tag("MyModel", "1", "approved_by", "ml-team-lead")
```

### 1.6 Model Lineage Tracking

The Model Registry automatically tracks provenance for each model:

#### Tracked Information
- **Source Run ID**: Which MLflow run produced the model
- **Source Experiment**: Parent experiment
- **Artifacts**: Associated files and data
- **Parameters**: Training configuration
- **Metrics**: Performance measurements
- **Code Version**: Git commit hash (if using MLflow Projects)

#### Accessing Lineage
```python
from mlflow import MlflowClient

client = MlflowClient()
version = client.get_model_version(name="MyModel", version="1")

print(f"Run ID: {version.run_id}")
print(f"Source: {version.source}")
print(f"Created: {version.creation_timestamp}")
```

### 1.7 Environment-Based Model Management

Modern pattern using separate registered models per environment:

```python
# Development
mlflow.register_model("runs:/<run-id>/model", "dev.ml_team.revenue_forecasting")

# Staging
mlflow.register_model("runs:/<run-id>/model", "staging.ml_team.revenue_forecasting")

# Production
mlflow.register_model("runs:/<run-id>/model", "prod.ml_team.revenue_forecasting")
```

#### Promoting Models Between Environments
```python
client.copy_model_version(
    src_model_uri="models:/dev.ml_team.revenue_forecasting/1",
    dst_name="staging.ml_team.revenue_forecasting"
)
```

---

## 2. MLflow Models Format

An MLflow Model is a standard format for packaging machine learning models that can be used across various deployment tools.

### 2.1 Model Directory Structure

```
my_model/
├── MLmodel                    # Model metadata (YAML)
├── model.pkl                  # Model artifact (flavor-specific)
├── conda.yaml                 # Conda environment
├── python_env.yaml            # Python environment
├── requirements.txt           # Pip dependencies
├── input_example.json         # Sample input
├── serving_input_example.json # REST API payload example
├── environment_variables.txt  # Required env var names
└── metadata/                  # Additional metadata
```

### 2.2 MLmodel File Structure

The `MLmodel` file is a YAML configuration defining how to use the model:

```yaml
artifact_path: model
flavors:
  python_function:
    env:
      conda: conda.yaml
      virtualenv: python_env.yaml
    loader_module: mlflow.sklearn
    model_path: model.pkl
    predict_fn: predict
    python_version: 3.10.12
  sklearn:
    code: null
    pickled_model: model.pkl
    serialization_format: cloudpickle
    sklearn_version: 1.3.2
mlflow_version: 2.9.2
model_size_bytes: 123456
model_uuid: abc123def456
run_id: 1234567890abcdef
signature:
  inputs: '[{"type": "double", "name": "feature1"}, {"type": "double", "name": "feature2"}]'
  outputs: '[{"type": "long", "name": "prediction"}]'
utc_time_created: '2024-01-15 10:30:00.000000'
```

### 2.3 Model Flavors

Flavors enable deployment tools to understand models from any ML library without specific integrations.

#### Built-in Flavors

**Traditional ML**
- `sklearn` - Scikit-learn models
- `xgboost` - XGBoost models
- `lightgbm` - LightGBM models
- `catboost` - CatBoost models
- `spark` - Spark MLlib models

**Deep Learning**
- `pytorch` - PyTorch models
- `tensorflow` - TensorFlow models
- `keras` - Keras models
- `onnx` - ONNX models

**LLM/GenAI**
- `transformers` - Hugging Face models
- `langchain` - LangChain applications
- `openai` - OpenAI API models
- `sentence_transformers` - Embedding models
- `llama_index` - LlamaIndex applications

**Time Series**
- `prophet` - Facebook Prophet
- `pmdarima` - Statistical time series
- `statsmodels` - Statsmodels

**Other**
- `spacy` - spaCy NLP
- `h2o` - H2O models
- `diviner` - Grouped time series

#### Universal python_function Flavor

All flavors include the `python_function` flavor, enabling generic inference:

```python
import mlflow.pyfunc

# Load any model as PyFunc
model = mlflow.pyfunc.load_model("models:/MyModel/1")
predictions = model.predict(data)
```

### 2.4 Model Signatures

Signatures define expected input/output schemas, providing validation during inference.

#### Column-Based Signatures (Tabular Data)

```python
from mlflow.models.signature import infer_signature
from mlflow.types.schema import Schema, ColSpec

# Automatic inference
signature = infer_signature(X_train, model.predict(X_train))

# Manual definition
input_schema = Schema([
    ColSpec("double", "sepal_length"),
    ColSpec("double", "sepal_width"),
    ColSpec("string", "species", required=False)
])
output_schema = Schema([ColSpec("long", "prediction")])
signature = ModelSignature(inputs=input_schema, outputs=output_schema)

# Log with signature
mlflow.sklearn.log_model(model, "model", signature=signature)
```

#### Tensor-Based Signatures (Deep Learning)

```python
from mlflow.types.schema import Schema, TensorSpec
import numpy as np

# Image classification model
input_schema = Schema([
    TensorSpec(np.dtype(np.float32), (-1, 28, 28, 1), "images")
])
output_schema = Schema([
    TensorSpec(np.dtype(np.float32), (-1, 10), "probabilities")
])
```

Note: `-1` indicates variable batch size.

#### Signature Enforcement

MLflow automatically validates inputs when:
- Loading models as PyFunc
- Using MLflow deployment tools
- Serving via REST API

**Important**: Signatures are REQUIRED for registering models in Databricks Unity Catalog.

### 2.5 Input Examples

Input examples provide concrete samples of valid model input:

```python
import pandas as pd

input_example = pd.DataFrame({
    "sepal_length": [5.1],
    "sepal_width": [3.5],
    "petal_length": [1.4],
    "petal_width": [0.2]
})

mlflow.sklearn.log_model(
    model,
    "model",
    input_example=input_example  # Signature auto-inferred
)
```

Benefits:
- Automatic signature inference
- Documentation for users
- Auto-generated serving payloads

### 2.6 Custom PyFunc Models

Create custom models by extending `mlflow.pyfunc.PythonModel`:

#### Basic Custom Model

```python
import mlflow.pyfunc

class CustomModel(mlflow.pyfunc.PythonModel):
    def load_context(self, context):
        """Load artifacts during initialization."""
        import pickle
        with open(context.artifacts["preprocessor"], "rb") as f:
            self.preprocessor = pickle.load(f)
        with open(context.artifacts["model"], "rb") as f:
            self.model = pickle.load(f)

    def predict(self, context, model_input, params=None):
        """Generate predictions."""
        processed = self.preprocessor.transform(model_input)
        return self.model.predict(processed)

# Log custom model
artifacts = {
    "preprocessor": "path/to/preprocessor.pkl",
    "model": "path/to/model.pkl"
}

mlflow.pyfunc.log_model(
    artifact_path="model",
    python_model=CustomModel(),
    artifacts=artifacts,
    conda_env="conda.yaml"
)
```

#### Streaming Predictions

```python
class StreamingModel(mlflow.pyfunc.PythonModel):
    def predict(self, context, model_input, params=None):
        return self._generate_all(model_input)

    def predict_stream(self, context, model_input, params=None):
        """Generator for streaming responses."""
        for chunk in self._generate_chunks(model_input):
            yield chunk
```

#### Type Hints for Validation

```python
from typing import List, Dict, Any

class TypedModel(mlflow.pyfunc.PythonModel):
    def predict(
        self,
        context,
        model_input: List[Dict[str, Any]],
        params: Dict[str, str] = None
    ) -> List[str]:
        # Automatic validation enabled
        return [self._process(item) for item in model_input]
```

### 2.7 Models From Code (MLflow 2.12.2+)

Define models in Python scripts without pickling:

```python
# my_model.py
import mlflow

class MyModel(mlflow.pyfunc.PythonModel):
    def predict(self, context, model_input, params=None):
        return model_input * 2

mlflow.models.set_model(MyModel())
```

```python
# Log the script
mlflow.pyfunc.log_model(
    artifact_path="model",
    python_model="my_model.py"
)
```

Benefits:
- Avoids pickle/cloudpickle security risks
- Cleaner code organization
- Better for complex dependencies
- Currently supports: LangChain, LlamaIndex, PythonModel

---

## 3. LLM-Specific Model Flavors

MLflow provides native flavors for popular LLM frameworks, introduced starting in MLflow 2.3.

### 3.1 mlflow.transformers

Integration with Hugging Face Transformers for experiment tracking, model management, and deployment.

#### Logging Models

```python
import mlflow
from transformers import pipeline

# Text generation pipeline
generator = pipeline("text-generation", model="gpt2")

with mlflow.start_run():
    mlflow.transformers.log_model(
        transformers_model=generator,
        artifact_path="generator",
        task="text-generation",
        registered_model_name="GPT2Generator"
    )
```

#### Supported Task Types

- `text-generation` - Text completion
- `text-classification` - Sentiment, classification
- `question-answering` - Q&A
- `summarization` - Text summarization
- `translation` - Language translation
- `fill-mask` - Masked language modeling
- `feature-extraction` - Embeddings

#### OpenAI-Compatible Chat (MLflow 2.11+)

```python
mlflow.transformers.log_model(
    transformers_model=model,
    artifact_path="chat_model",
    task="llm/v1/chat"  # OpenAI-compatible endpoint
)
```

#### Component Logging

```python
# Log individual components
mlflow.transformers.log_model(
    transformers_model={
        "model": model,
        "tokenizer": tokenizer,
        "model_config": config
    },
    artifact_path="model"
)
```

#### PEFT/LoRA Support

```python
from peft import LoraConfig, get_peft_model

# Native support for parameter-efficient fine-tuning
peft_model = get_peft_model(base_model, lora_config)
mlflow.transformers.log_model(peft_model, "lora_model")
```

#### Loading Methods

```python
# Native Transformers format
model = mlflow.transformers.load_model("models:/MyModel/1")

# PyFunc for deployment
pyfunc_model = mlflow.pyfunc.load_model("models:/MyModel/1")
predictions = pyfunc_model.predict(data)
```

### 3.2 mlflow.langchain

Integration for LangChain applications including chains, agents, and retrievers.

#### Supported Components

- **Agents**: Dynamic constructs using LLMs for action sequences
- **Retrievers**: Document sourcing for RAG applications
- **Runnables**: LangChain Expression Language (LCEL) components
- **LangGraph**: Stateful multi-agent applications

#### Logging Chains

```python
import mlflow
from langchain.chains import LLMChain

with mlflow.start_run():
    mlflow.langchain.log_model(
        lc_model=chain,
        artifact_path="chain",
        registered_model_name="MyChain"
    )
```

#### Autologging

```python
mlflow.langchain.autolog()

# All chain executions automatically logged
result = chain.invoke({"question": "What is MLflow?"})
```

#### Models-from-Code (Recommended)

For complex chains with unpicklable components:

```python
# chain_definition.py
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
import mlflow

prompt = ChatPromptTemplate.from_template("Tell me about {topic}")
chain = prompt | ChatOpenAI()

mlflow.models.set_model(chain)
```

```python
# Log the script
mlflow.langchain.log_model(
    lc_model="chain_definition.py",
    artifact_path="chain"
)
```

#### RAG Applications

```python
from langchain.chains import RetrievalQA
from langchain_community.vectorstores import FAISS

# Create retriever
vectorstore = FAISS.from_documents(documents, embeddings)
retriever = vectorstore.as_retriever()

# Create RAG chain
rag_chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=retriever
)

mlflow.langchain.log_model(rag_chain, "rag_chain")
```

#### Streaming Support (MLflow 2.12.2+)

```python
model = mlflow.pyfunc.load_model("models:/MyChain/1")

for chunk in model.predict_stream({"question": "Explain MLOps"}):
    print(chunk, end="")
```

### 3.3 mlflow.openai

Integration for OpenAI API applications.

#### Logging OpenAI Models

```python
import mlflow.openai

with mlflow.start_run():
    mlflow.openai.log_model(
        model="gpt-4",
        task="chat.completions",
        artifact_path="openai_model",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."}
        ]
    )
```

#### Task Types

- `chat.completions` - Conversational AI
- `completions` - Text completion (legacy)
- `embeddings` - Text embeddings

#### Autologging and Tracing

```python
mlflow.openai.autolog()

from openai import OpenAI
client = OpenAI()

# Automatically traced
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

#### Function Calling

```python
functions = [
    {
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            }
        }
    }
]

mlflow.openai.log_model(
    model="gpt-4",
    task="chat.completions",
    artifact_path="function_model",
    functions=functions
)
```

### 3.4 mlflow.sentence_transformers

Integration for Sentence Transformers embedding models.

#### Logging Embedding Models

```python
from sentence_transformers import SentenceTransformer
import mlflow

model = SentenceTransformer("all-MiniLM-L6-v2")

with mlflow.start_run():
    mlflow.sentence_transformers.log_model(
        model,
        artifact_path="embedder",
        registered_model_name="TextEmbedder"
    )
```

#### OpenAI-Compatible Embeddings

```python
mlflow.sentence_transformers.log_model(
    model,
    artifact_path="embedder",
    task="llm/v1/embeddings"  # OpenAI-compatible format
)
```

#### Loading and Inference

```python
# Native loading
model = mlflow.sentence_transformers.load_model("models:/TextEmbedder/1")
embeddings = model.encode(["Hello world", "How are you?"])

# PyFunc loading
pyfunc_model = mlflow.pyfunc.load_model("models:/TextEmbedder/1")
embeddings = pyfunc_model.predict(["Hello world"])
```

#### Inference Configuration

```python
mlflow.sentence_transformers.log_model(
    model,
    artifact_path="embedder",
    inference_config={
        "batch_size": 32,
        "normalize_embeddings": True
    }
)
```

#### Serving

```bash
mlflow models serve -m 'models:/TextEmbedder/1' -p 5000
```

```bash
curl -X POST http://localhost:5000/invocations \
  -H "Content-Type: application/json" \
  -d '{"inputs": ["Hello world"]}'
```

### 3.5 mlflow.llama_index

Integration for LlamaIndex RAG applications and workflows.

#### Supported Objects

- **Index**: Vector stores, knowledge graphs
- **Query Engine**: Straightforward retrieval
- **Chat Engine**: Conversational with context
- **Retriever**: Document retrieval
- **Workflow**: Event-driven orchestration (MLflow 2.15+)

#### Logging LlamaIndex Applications

```python
import mlflow
from llama_index.core import VectorStoreIndex

# Create index
index = VectorStoreIndex.from_documents(documents)

with mlflow.start_run():
    mlflow.llama_index.log_model(
        llama_index_model=index,
        artifact_path="index",
        engine_type="query",  # or "chat", "retriever"
        registered_model_name="RAGIndex"
    )
```

#### Engine Types

```python
# Query engine - simple retrieval
mlflow.llama_index.log_model(index, "model", engine_type="query")

# Chat engine - conversational with history
mlflow.llama_index.log_model(index, "model", engine_type="chat")

# Retriever - document retrieval only
mlflow.llama_index.log_model(index, "model", engine_type="retriever")
```

#### Workflow Support

```python
from llama_index.core.workflow import Workflow

class RAGWorkflow(Workflow):
    @step
    async def retrieve(self, ev: StartEvent) -> RetrieveEvent:
        # Retrieval logic
        pass

    @step
    async def synthesize(self, ev: RetrieveEvent) -> StopEvent:
        # Generation logic
        pass

workflow = RAGWorkflow()
mlflow.llama_index.log_model(workflow, "workflow")
```

#### Autologging and Tracing

```python
mlflow.llama_index.autolog()

# Nested traces automatically logged
response = query_engine.query("What is MLflow?")
```

#### Model Configuration

```python
mlflow.llama_index.log_model(
    index,
    artifact_path="index",
    engine_type="chat",
    model_config={
        "temperature": 0.7,
        "system_prompt": "You are a helpful assistant."
    }
)
```

### 3.6 ChatModel and ResponsesAgent

#### ChatModel (MLflow 2.11+)

Standardized interface for conversational AI, compatible with OpenAI's ChatCompletion API:

```python
import mlflow
from mlflow.pyfunc import ChatModel
from mlflow.types.llm import ChatMessage, ChatResponse

class MyChatModel(ChatModel):
    def predict(self, context, messages, params=None):
        # Process messages
        response_text = self._generate_response(messages)
        return ChatResponse(
            choices=[{
                "message": ChatMessage(
                    role="assistant",
                    content=response_text
                )
            }]
        )

mlflow.pyfunc.log_model(
    artifact_path="chat",
    python_model=MyChatModel()
)
```

#### ResponsesAgent (MLflow 3.0+)

Recommended for new projects:

```python
from mlflow.pyfunc import ResponsesAgent

class MyAgent(ResponsesAgent):
    def predict(self, context, input_data, params=None):
        # Handle tool calls, streaming, etc.
        pass

mlflow.pyfunc.log_model(
    artifact_path="agent",
    python_model=MyAgent()
)
```

Features:
- Full OpenAI API compatibility
- Streaming support
- Tool call tracking
- Usage metrics

---

## 4. Model Serving and Deployment

MLflow provides a unified interface for deploying models to various targets.

### 4.1 Local Model Serving

#### Basic Serving

```bash
mlflow models serve -m runs:/<run-id>/model -p 5000
```

Or from Model Registry:

```bash
mlflow models serve -m "models:/MyModel/1" -p 5000
mlflow models serve -m "models:/MyModel@champion" -p 5000
```

#### REST API Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/invocations` | POST | Generate predictions |
| `/ping` | GET | Health check |
| `/health` | GET | Health check |
| `/version` | GET | MLflow version |

#### Request Formats

**JSON (DataFrame Split)**
```bash
curl -X POST http://localhost:5000/invocations \
  -H "Content-Type: application/json" \
  -d '{
    "dataframe_split": {
      "columns": ["feature1", "feature2"],
      "data": [[1.0, 2.0], [3.0, 4.0]]
    }
  }'
```

**JSON (Instances)**
```bash
curl -X POST http://localhost:5000/invocations \
  -H "Content-Type: application/json" \
  -d '{
    "instances": [
      {"feature1": 1.0, "feature2": 2.0}
    ]
  }'
```

**CSV**
```bash
curl -X POST http://localhost:5000/invocations \
  -H "Content-Type: text/csv" \
  -d 'feature1,feature2
1.0,2.0
3.0,4.0'
```

#### Serving Options

```bash
# Specify environment manager
mlflow models serve -m model_uri --env-manager virtualenv

# Enable MLServer (high performance)
mlflow models serve -m model_uri --enable-mlserver
```

### 4.2 Batch Inference

```bash
mlflow models predict \
  -m "models:/MyModel/1" \
  -i input.csv \
  -o output.csv
```

```python
# Python API
import mlflow.pyfunc

model = mlflow.pyfunc.load_model("models:/MyModel/1")
predictions = model.predict(input_data)
```

### 4.3 Docker Containerization

#### Building Docker Images

```bash
# Basic Docker image
mlflow models build-docker \
  -m "models:/MyModel/1" \
  -n my-model-image

# With MLServer for Kubernetes
mlflow models build-docker \
  -m "models:/MyModel/1" \
  -n my-model-image \
  --enable-mlserver
```

#### Running Container

```bash
docker run -p 5000:8080 my-model-image
```

#### Custom Dockerfile

```dockerfile
FROM python:3.10-slim

RUN pip install mlflow

COPY model /model

CMD ["mlflow", "models", "serve", "-m", "/model", "-h", "0.0.0.0", "-p", "8080"]
```

### 4.4 Cloud Platform Deployments

#### Amazon SageMaker

```python
import mlflow.sagemaker

mlflow.sagemaker.deploy(
    app_name="my-model",
    model_uri="models:/MyModel/1",
    region_name="us-west-2",
    mode="create",
    instance_type="ml.m5.large"
)
```

#### Azure ML

```python
from mlflow.deployments import get_deploy_client

client = get_deploy_client("azureml://...")

client.create_deployment(
    name="my-deployment",
    model_uri="models:/MyModel/1",
    config={
        "instance_type": "Standard_DS3_v2",
        "instance_count": 1
    }
)
```

#### Databricks Model Serving

```python
from mlflow.deployments import get_deploy_client

client = get_deploy_client("databricks")

client.create_endpoint(
    name="my-endpoint",
    config={
        "served_models": [{
            "model_name": "MyModel",
            "model_version": "1",
            "workload_size": "Small",
            "scale_to_zero_enabled": True
        }]
    }
)
```

### 4.5 Kubernetes Deployment

#### KServe/Seldon Core

```bash
# Build MLServer-enabled image
mlflow models build-docker \
  -m "models:/MyModel/1" \
  -n my-model:latest \
  --enable-mlserver

# Push to registry
docker push my-registry/my-model:latest
```

**KServe InferenceService:**

```yaml
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: my-model
spec:
  predictor:
    containers:
      - name: mlserver
        image: my-registry/my-model:latest
        ports:
          - containerPort: 8080
        resources:
          requests:
            memory: "2Gi"
            cpu: "1"
```

**Kubernetes Deployment:**

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mlflow-model
spec:
  replicas: 3
  selector:
    matchLabels:
      app: mlflow-model
  template:
    metadata:
      labels:
        app: mlflow-model
    spec:
      containers:
        - name: model
          image: my-registry/my-model:latest
          ports:
            - containerPort: 8080
          resources:
            requests:
              memory: "2Gi"
              cpu: "1"
---
apiVersion: v1
kind: Service
metadata:
  name: mlflow-model-service
spec:
  selector:
    app: mlflow-model
  ports:
    - port: 80
      targetPort: 8080
  type: LoadBalancer
```

### 4.6 Inference Server Options

#### FastAPI (Default)

- Included with MLflow
- Asynchronous request handling
- Suitable for most use cases

#### MLServer (High Performance)

```bash
pip install mlflow[extras]
mlflow models serve -m model_uri --enable-mlserver
```

Benefits:
- Adaptive batching
- Parallel inference
- V2 Inference Protocol
- KServe/Seldon integration

### 4.7 MLflow AI Gateway

Unified endpoint for managing multiple LLM providers.

#### Installation

```bash
pip install 'mlflow[gateway]'
```

#### Configuration

```yaml
# gateway_config.yaml
endpoints:
  - name: chat
    endpoint_type: llm/v1/chat
    model:
      provider: openai
      name: gpt-4
      config:
        openai_api_key: $OPENAI_API_KEY
    limit:
      renewal_period: minute
      calls: 100

  - name: embeddings
    endpoint_type: llm/v1/embeddings
    model:
      provider: openai
      name: text-embedding-3-small
      config:
        openai_api_key: $OPENAI_API_KEY
```

#### Starting Gateway

```bash
mlflow gateway start --config-path gateway_config.yaml --port 7000
```

#### Supported Providers

- OpenAI (including Azure OpenAI)
- Anthropic
- Cohere
- Amazon Bedrock
- Mistral
- AI21 Labs
- MosaicML
- Hugging Face TGI

#### Benefits

- Centralized API key management
- Rate limiting
- Provider swapping without code changes
- Consistent API across providers
- Hot-swapping endpoints (no restart required)

---

## 5. Model Governance

MLflow provides capabilities for managing model lifecycle, approvals, and access control.

### 5.1 Model Approval Workflows

#### Using Tags for Status Tracking

```python
from mlflow import MlflowClient

client = MlflowClient()

# Mark for review
client.set_model_version_tag(
    name="MyModel",
    version="1",
    key="approval_status",
    value="pending_review"
)

# Record reviewer
client.set_model_version_tag(
    name="MyModel",
    version="1",
    key="reviewed_by",
    value="ml-team-lead"
)

# Mark approved
client.set_model_version_tag(
    name="MyModel",
    version="1",
    key="approval_status",
    value="approved"
)

# Promote to production
client.set_registered_model_alias(
    name="MyModel",
    alias="champion",
    version=1
)
```

#### CI/CD Integration

```python
# Example: Jenkins/GitHub Actions workflow
def validate_and_promote_model(model_name, version):
    client = MlflowClient()

    # Load model
    model = mlflow.pyfunc.load_model(f"models:/{model_name}/{version}")

    # Run tests
    test_results = run_model_tests(model)

    if test_results.passed:
        # Update tags
        client.set_model_version_tag(
            model_name, version,
            "validation_status", "passed"
        )

        # Promote to staging
        client.copy_model_version(
            f"models:/{model_name}/{version}",
            f"staging.{model_name}"
        )

        return True
    else:
        client.set_model_version_tag(
            model_name, version,
            "validation_status", "failed"
        )
        return False
```

### 5.2 Audit Trails

#### Automatic Tracking

MLflow automatically records:
- **Creation timestamps**: When models were registered
- **Source run IDs**: Training provenance
- **User information**: Who registered the model (with auth enabled)
- **Tag history**: All metadata changes

#### Accessing Audit Information

```python
from mlflow import MlflowClient

client = MlflowClient()

# Get model version details
version = client.get_model_version(name="MyModel", version="1")

print(f"Created: {version.creation_timestamp}")
print(f"Last Updated: {version.last_updated_timestamp}")
print(f"Source Run: {version.run_id}")
print(f"User: {version.user_id}")  # With auth enabled

# Get all tags
for tag in version.tags:
    print(f"{tag.key}: {tag.value}")
```

#### Lineage Queries

```python
# Get complete model history
versions = client.search_model_versions(f"name='{model_name}'")

for version in versions:
    print(f"Version {version.version}:")
    print(f"  Run ID: {version.run_id}")
    print(f"  Created: {version.creation_timestamp}")
    print(f"  Status: {version.status}")
```

### 5.3 Access Control

#### MLflow Authentication (Open Source)

Enable basic HTTP authentication:

```bash
pip install mlflow[auth]
export MLFLOW_FLASK_SERVER_SECRET_KEY="secret-key"
mlflow server --app-name basic-auth
```

#### Permission Levels

| Permission | Capabilities |
|------------|-------------|
| `READ` | View access only |
| `EDIT` | Read and modify |
| `MANAGE` | Full control including permissions |
| `NO_PERMISSIONS` | No access |

#### Managing Permissions

```python
from mlflow.server import get_app_client

auth_client = get_app_client(
    "basic-auth",
    tracking_uri="http://localhost:5000/"
)

# Create user
auth_client.create_user(username="analyst", password="password123")

# Grant experiment access
auth_client.create_experiment_permission(
    experiment_id="1",
    username="analyst",
    permission="EDIT"
)

# Grant model access
auth_client.create_registered_model_permission(
    name="MyModel",
    username="analyst",
    permission="READ"
)
```

#### Admin Users

Default admin credentials (change immediately):
- Username: `admin`
- Password: `password`

Admin capabilities:
- Create/delete users
- Update passwords and admin status
- Grant/revoke all permissions
- Access all resources

#### Authentication Methods

1. **UI Login**: Prompted on first visit

2. **Environment Variables**:
```bash
export MLFLOW_TRACKING_USERNAME=myuser
export MLFLOW_TRACKING_PASSWORD=mypassword
```

3. **Credentials File** (`~/.mlflow/credentials`):
```ini
[mlflow]
mlflow_tracking_username = myuser
mlflow_tracking_password = mypassword
```

4. **REST API**:
```bash
curl -u myuser:mypassword http://localhost:5000/api/2.0/mlflow/experiments/list
```

### 5.4 Enterprise Governance Solutions

#### Databricks Unity Catalog

Features:
- Centralized governance across workspaces
- Fine-grained access control
- Cross-workspace model discovery
- Integrated lineage tracking
- Data masking and encryption

```python
# Register to Unity Catalog
mlflow.set_registry_uri("databricks-uc")

mlflow.sklearn.log_model(
    model,
    "model",
    registered_model_name="main.ml_models.my_model"
)
```

#### AWS Integration

Use API Gateway for enterprise access control:
- Separate authentication/authorization from MLflow server
- IAM role-based access
- Fine-grained permissions via policies

#### Azure ML RBAC

Leverage Azure's role-based access control:
- Built-in roles (ML Scientist, ML Engineer)
- Custom role definitions
- Resource group scoping

### 5.5 Security Best Practices

1. **API Key Management**
   - Never commit API keys to code
   - Use environment variables
   - Rotate keys regularly
   - MLflow logs env var names (not values) in `environment_variables.txt`

2. **Model Validation**
   - Always validate models before promotion
   - Use `mlflow.models.predict()` for pre-deployment testing
   - Implement automated testing pipelines

3. **Environment Separation**
   - Use separate registered models per environment
   - Apply appropriate access controls per environment
   - Document promotion requirements

4. **Audit and Compliance**
   - Enable authentication for tracking
   - Use tags to document approvals
   - Maintain lineage for reproducibility
   - Regular access reviews

5. **Network Security**
   - Use HTTPS for all communications
   - Implement network isolation where needed
   - Consider VPN for remote access

---

## Appendix: Quick Reference

### Common MLflow Commands

```bash
# Start tracking server
mlflow server --host 0.0.0.0 --port 5000

# With authentication
mlflow server --app-name basic-auth

# Serve model locally
mlflow models serve -m "models:/MyModel/1" -p 5000

# Build Docker image
mlflow models build-docker -m "models:/MyModel/1" -n my-image

# Batch prediction
mlflow models predict -m "models:/MyModel/1" -i input.csv -o output.csv
```

### Common Python Patterns

```python
import mlflow
from mlflow import MlflowClient

# Set tracking URI
mlflow.set_tracking_uri("http://localhost:5000")

# Start run and log model
with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.01)
    mlflow.log_metric("accuracy", 0.95)
    mlflow.sklearn.log_model(
        model, "model",
        registered_model_name="MyModel"
    )

# Load model for inference
model = mlflow.pyfunc.load_model("models:/MyModel@champion")
predictions = model.predict(data)

# Client operations
client = MlflowClient()
client.set_registered_model_alias("MyModel", "champion", 1)
client.set_model_version_tag("MyModel", "1", "status", "approved")
```

### Model URI Formats

| Format | Example |
|--------|---------|
| Run artifact | `runs:/<run-id>/model` |
| Registry version | `models:/<name>/<version>` |
| Registry alias | `models:/<name>@<alias>` |
| S3 | `s3://bucket/path/to/model` |
| Local file | `file:///path/to/model` |

### Environment Variables

| Variable | Description |
|----------|-------------|
| `MLFLOW_TRACKING_URI` | Tracking server URL |
| `MLFLOW_REGISTRY_URI` | Model registry URL |
| `MLFLOW_TRACKING_USERNAME` | Auth username |
| `MLFLOW_TRACKING_PASSWORD` | Auth password |
| `MLFLOW_AUTH_CONFIG_PATH` | Auth config file path |

---

## References

- [MLflow Official Documentation](https://mlflow.org/docs/latest/)
- [MLflow Model Registry](https://mlflow.org/docs/latest/ml/model-registry/)
- [MLflow GenAI Documentation](https://mlflow.org/docs/latest/genai/)
- [MLflow Deployment Guide](https://mlflow.org/docs/latest/ml/deployment/)
- [MLflow Authentication](https://mlflow.org/docs/latest/self-hosting/security/basic-http-auth/)
- [MLflow GitHub Repository](https://github.com/mlflow/mlflow)
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall