Demos

Horizontal Pod Autoscaling - to cope with rise/fall in requests to meet service level requirements. Time: 2.07 minutes.

 

TGI-HPA-eng-20241114.mp4

 

On a single node using Docker Engine, running in Intel Tiber Cloud

 

 

 

On Kubernetes running anywhere using OPEA GenAI Microservice Connector (GMC). 

The demo shows response to a prompt requesting current information, ingesting current data to answer the prompt and finally changing the model.

Using GMC supports re-use of unchanged services when changing/updating a GenAI pipeline definition, which is particularly valuable during the development phase.

Further GMC supports re-use of common services in the same namespace between distinct GenAI applications/pipelines

making things more resource efficient through sharing and faster to launch.

 

 

 

This page highlights different demos that can be shown for OPEA