Including code files

Remote file from GitHub

A GitHub page URL is rewritten to the raw file. The language comes from the .yaml extension.

apiVersion: serving.kserve.io/v1alpha2
kind: LLMInferenceServiceConfig
metadata:
  name: kserve-config-llm-scheduler-eppconfig-default
  namespace: kserve
spec:
  router:
    scheduler:
      config:
        # Default single-profile EndpointPickerConfig.
        # lora-affinity-scorer is injected by the scheduler when the LLMInferenceService has spec.model.lora.adapters set.
        inline:
          apiVersion: llm-d.ai/v1alpha1
          kind: EndpointPickerConfig
          plugins:
            - type: approx-prefix-cache-producer
            - type: inflight-load-producer
            - type: prefix-cache-affinity-filter
            - type: token-load-scorer
          schedulingProfiles:
            - name: default
              plugins:
                - pluginRef: prefix-cache-affinity-filter
                - pluginRef: token-load-scorer

Selected lines of a remote file

GitHub-style #L13-L19 fragments select lines; normalize-space removes the shared indentation.

apiVersion: llm-d.ai/v1alpha1
kind: EndpointPickerConfig
plugins:
  - type: approx-prefix-cache-producer
  - type: inflight-load-producer
  - type: prefix-cache-affinity-filter
  - type: token-load-scorer

Local files

apiVersion: serving.kserve.io/v1alpha2
kind: LLMInferenceServiceConfig
metadata:
  name: kserve-config-llm-scheduler-eppconfig-default
  namespace: kserve
spec:
  router:
    scheduler:
      config:

{
  "name": "demo", "replicas": 3, "enabled": true, "ratio": -1.5e3,
  "tags": ["a", "b\"c"], "owner": null
}

#!/usr/bin/env bash
# Deploy the config
set -euo pipefail
NS="${1:-kserve}"
for f in config/*.yaml; do
  kubectl apply -n "$NS" -f "$f" --server-side   # apply
done
echo 'done' $HOME

Forcing a language

An explicit outputclass="language-…" wins over the file name. This also works for inline code with no coderef.

replicas: 3   # inline YAML
selector: {app: picker, tier: "edge"}