feat: upgrade vllm backend and refactor deployment

[justinthelaw]

OVERVIEW

See #835 for more details on rationale and findings. Also related to #623.

IMPORTANT NOTE: there are still ongoing AsyncLLMEngineDead and RoPE scaling + Ray issues upstream that may prevent us from upgrading past 0.4.x.

BREAKING CHANGES:

• moves all ENV specific to LeapfrogAI SDK to a ConfigMap using volumeMount for runtime injection and modification
  • in local dev, this is defined via config.yaml
• moves all ENV specific to vLLM to a ConfigMap, using envFrom for runtime injection and modification
  • in local dev, this is defined via .env
• ZARF_CONFIG is used to define create-time adn deploy-time variables for (e.g., MODEL_REPO_ID, ENFORCE_EAGER)
  • allows delivery engineer's declaritive definition of the backend configs and model
• re-introduces LFAI SDK config.yaml configuration method for local development and testing

CHANGES:

• updates docs for running vLLM locally
• update to Python 3.11.9 to align with Registry1 base image
• upgrades vLLM from 0.4.2 to 0.4.3 for:
  • adds BFloat16 quantized model support (the "why is this important" here)
• exposes more backend configurations for Zarf build (see packages/vllm/README.md)
  • adds full set of QUANTIZATION options to existing configuration field
  • exposes everything via a Zarf variable and the values files
• removes default vLLM engine configurations from the Dockerfile (no duplicates, hardcoding)
• fixes issue where request params are not used in generation of the response (e.g., temperature)
  • uses LFAI SDK-received request object for inferencing params
  • gracefully handles vLLM versus SDK param names/differences for generation

ADDITIONAL CONTEXT

The default model is still Synthia-7b, until #976 is resolved. The below description is only being kept for future context:

The new model option, defenseunicorns/Hermes-2-Pro-Mistral-7B-4bit-32g, takes ~4.16Gb to load on to RAM or vRAM. At 6Gb of vRAM, the max_context_length variable has to be reduced to ~400 tokens; at 8Gb vRAM, ~10K tokens; and at 12Gb of vRAM, ~15K tokens. All cases require the max_gpu_utilization variable to be 0.99 in order to max out the KV cache size for the context length that is to be reserved at inference.

To achieve the maximum context length (~32K tokens) of the model, 16Gb of vRAM is required.

Reaching the defined max_context_length during a completion or chatting will result in the activation of vLLM's automatic sliding window handler, which drops quality of the final responses significantly.

[justinthelaw]

Blocked by #1038, wait until that is merged and re-test this branch locally with the new vLLM E2E.