Inference Deployment

The below component allows realtime inference on the model of your choosing. The models are created as ONNX snapshots and inference is performed in an AWS Lambda. How is inference possible in such a constrained environment? Because the models are very small.

Tinystories

The models were trained on the 'tinystories' dataset, a synthetic dataset created for just this purpose. Featuring simple syntax and limited vocabulary, even small Transformers can effectively learn the data. Here is an example:

Emily was very excited. She was going to the office to paint. She loved painting, and she had heard the office was very interesting. When she arrived, Emily couldn't believe her eyes! There was so much to paint - big walls, small walls, and even a ceiling. Emily was in heaven! She spent the whole day painting with bright colors and own ideas. At the end of the day, Emily was exhausted but very happy. She had painted her very own masterpiece! Every time she looked at it, she was filled with pride. The office was now full of wonderful pictures that she had created. Emily couldn't wait to go back and paint again. She knew that each time she returned, the office would be even more interesting!

This should frame your expectations on what model outputs will look like. Happy inferring!