Harnessing the power of Embeddings: a real-world example of a watch recommendation app using a vector database.

May 20, 2024·

·Reading time: 19 minutes

Given the increasing integration of Machine Learning and AI/LLM models in all applications, it is crucial to explore the potential of a robust cloud application platform like Upsun for conducting these experiments.

As a watch enthusiast, what would be better than a recommendation engine to find the right watch for you? Here is the app we are going to build from scratch here:

Watch Recommendation App

Source

You can find the full example repository on Github: https://github.com/gmoigneu/upsun-embeddings-watches

Architecture

Let’s review the architecture we are going to create and deploy:

Watch Recommendation Architecture

Our application will be composed of 3 main components:

A static React frontend is used to present the UI to the user.
An API built with Laravel 11 will handle the search requests and populate our database.
A Postgres instance to store our watches records and vectors for the embeddings of our content.

For speed and cost efficiency reasons, we will leverage both OpenAI for creating embeddings and Groq for the LLM part. We will use llama3–8b-8192 as a model on Groq, but mixtral or any other would also work.

Bootstrapping our apps

You will need php8.x and composer installed locally for this to work. As we deploy our app on Upsun, you can grab the upsun cli by following the steps here.

First, create a new folder for your project:

Terminal

mkdir watches && cd watches

Let’s start by installing Laravel 11 with the API components:

Terminal

composer create-project laravel/laravel watches
cd watches
php artisan api:install

You will need a few extra packages:

Terminal

# required 
composer require openai-php/client
composer require pgvector/pgvector

# optional
composer require laravel/telescope # for debugging requests
composer require spatie/laravel-ray # amazing little debugging helper

Let’s now create a new React app. We are using Vite here with the react-ts template but feel free to use any other variant. We will set up shacdn/ui for the UI to speed up building our interface.

Terminal

cd ..
npm create vite@latest frontend -- --template react-ts
cd frontend

# Install shacdn/ui
npm create vite@latest
npm install -D tailwindcss postcss autoprefixer

npx tailwindcss init -p

# (so you can import "path" without error)
npm i -D @types/node

Add the following code to the tsconfig.json file to resolve paths:

tsconfig.json

{
  "compilerOptions": {
    // ...
    "baseUrl": ".",
    "paths": {
      "@/*": [
        "./src/*"
      ]
    }
    // ...
  }
}

Add the following code to the vite.config.ts so your app can resolve paths without error

vite.config.ts

// vite.config.ts

import path from "path"
import react from "@vitejs/plugin-react"
import { defineConfig } from "vite"

export default defineConfig({
  plugins: [react()],
  resolve: {
    alias: {
      "@": path.resolve(__dirname, "./src"),
    },
  },
})

Now run the shacdn init with the following settings:

Terminal

npx shadcn-ui@latest init

Would you like to use TypeScript (recommended)? yes
Which style would you like to use? › Default
Which color would you like to use as base color? › Slate
Where is your global CSS file? › › src/index.css
Do you want to use CSS variables for colors? › yes
Where is your tailwind.config.js located? › tailwind.config.js
Configure the import alias for components: › @/components
Configure the import alias for utils: › @/lib/utils
Are you using React Server Components? › no (no)

Add the needed components for our UI:

Terminal

npx shadcn-ui@latest add button
npx shadcn-ui@latest add textarea

Our React app will use two extra packages:

Terminal

npm i @microsoft/fetch-event-source # AJAX requests with EventStream support
npm i axios # for making API requests

Our two apps are now bootstrapped. Time to initialize our git repo:

Terminal

cd ..
git init .
git add watches 
git commit -m "Add Laravel API to the repo"

git add frontend 
git commit -m "Add React frontend to the repo"

Setting up our local development environment & infrastructure

While you could use Laravel Sail or any other Docker-based local setup to develop the project, I find relying on the Upsun tethering feature always way quicker. It maps the remote services (database, Redis, etc.) to local ports, so you only need to run the runtimes (PHP, JS, …) locally.

To do that, we first need to go through the Upsun configuration. While you can use the upsun ify CLI command to generate an automatic configuration, here is the complete configuration explained. Create a new .upsun/config.yaml and paste the following:

.upsun/config.yaml

# Routing for the frontend and API
routes:
  "https://frontend.{all}/": { type: upstream, upstream: "frontend:http" }
  "https://api.{all}/": { type: upstream, upstream: "app:http" }
  "http://{all}/": { type: redirect, to: "https://frontend.{all}/" }

services: 
  database:
    type: postgresql:15 # We will use Postgres with the vector extension
    configuration:
      extensions:
        - vector
  redis: # Redis is used for the Laravel cache
    type: redis:7.0
 
applications:
  # Our react frontend
  frontend:
    source:
      root: "frontend" # folder where the app is
    type: nodejs:20
    build:
      flavor: none
    web:
      locations:
        "/":
          root: "dist" # Customize if your build outputs the app somewhere else
          expires: 1h
          passthru: "/index.html"
      commands:
        start: sleep infinity # needed as we serve static files

    hooks: # customize if you use another build tool
      build: |
        set -x -e
        npm install 
        npm run build        
  app:
    source:
      root: "watches"

    type: php:8.3

    dependencies:
      php:
        composer/composer: "^2"

    runtime:
      extensions:
        - apcu
        - redis
        - blackfire
        - ctype
        - iconv
        - mbstring
        - pdo_pgsql
        - pdo_pgsql
        - sodium
        - xsl
    build:
      flavor: none
    web:
      locations:
        "/":
          root: "public"
          expires: 1h
          passthru: "/index.php"
    mounts: # Where Laravel needs write access
      "/.config":
        source: "storage"
        source_path: "config"
        
      "bootstrap/cache":
        source: "storage"
        source_path: "cache"
        
      "storage":
        source: "storage"
        source_path: "storage"
    relationships:
      # link our services
      database: "database:postgresql"
      redis: "redis:redis"
    hooks:
      build: |
        set -eux
        composer --no-ansi --no-interaction install --no-progress --prefer-dist --optimize-autoloader --no-dev        
      deploy: |
        # customize if needed but should be ok for Laravel
        set -eux
        mkdir -p storage/framework/sessions
        mkdir -p storage/framework/cache
        mkdir -p storage/framework/views
        php artisan migrate --force
        php artisan optimize:clear

Once done, we must inject some variables into our Laravel app to grab the proper credentials. Create the watches/.environment file with the following:

watches/.environment

# Set database environment variables
export DB_HOST="$DATABASE_HOST"
export DB_PORT="$DATABASE_PORT"
export DB_PATH="$DATABASE_PATH"
export DB_USERNAME="$DATABASE_USERNAME"
export DB_PASSWORD="$DATABASE_PASSWORD"
export DB_SCHEME="postgresql"
export DATABASE_URL="${DB_SCHEME}://${DB_USERNAME}:${DB_PASSWORD}@${DB_HOST}:${DB_PORT}/${DB_PATH}"

# Set Laravel-specific environment variables
export DB_CONNECTION="pgsql"
export DB_DATABASE="$DB_PATH"

# Set Cache environment variables
export CACHE_STORE="redis"
export CACHE_HOST="$REDIS_HOST"
export CACHE_PORT="$REDIS_PORT"
export CACHE_SCHEME="$REDIS_SCHEME"
export CACHE_URL="${CACHE_SCHEME}://${CACHE_HOST}:${CACHE_PORT}"

# Set Redis environment variables
export REDIS_URL="$CACHE_URL"

You can customize any other environment variable this way. Remember they will always be the same value as this is committed to our repo.

Let’s add the configuration to our git repo:

Terminal

git add .upsun
git add .environment
git commit -m "Add Upsun configuration"

Our first deploy

Let’s create an Upsun project for our application:

Terminal

upsun project:create

* Project title (--title)
Default: Untitled Project
> Watch finder

* Region (--region)
The region where the project will be hosted.
Get a 3% discount on resources for regions with a carbon intensity of less than 100 gCO2eq/kWh.
  [au.platform.sh  ] Sydney, Australia (AWS)  [545 gC02eq/kWh]
  [au-2.platform.sh] Sydney, Australia (AZURE)  [545 gC02eq/kWh]
  [ca-1.platform.sh] Montreal, Canada (AWS)  [31 gC02eq/kWh]
  [ch-1.platform.sh] Zurich, Switzerland (GCP)  [91 gC02eq/kWh]
  [de-2.platform.sh] Frankfurt, Germany (GCP)  [416 gC02eq/kWh]
  [eu.platform.sh  ] Dublin, Ireland (AWS)  [386 gC02eq/kWh]
  [eu-2.platform.sh] Dublin, Ireland (AWS)  [386 gC02eq/kWh]
  [eu-4.platform.sh] Dublin, Ireland (AWS)  [386 gC02eq/kWh]
  [eu-5.platform.sh] Stockholm, Sweden (AWS)  [23 gC02eq/kWh]
  [fr-3.platform.sh] Gravelines, France (OVH)  [59 gC02eq/kWh]
  [fr-4.platform.sh] Paris, France (AZURE)  [59 gC02eq/kWh]
  [uk-1.platform.sh] London, United Kingdom (GCP)  [200 gC02eq/kWh]
  [us.platform.sh  ] Washington, United States (AWS)  [396 gC02eq/kWh]
  [us-2.platform.sh] Washington, United States (AWS)  [396 gC02eq/kWh]
  [us-3.platform.sh] Moses Lake, United States (AZURE)  [56 gC02eq/kWh]
  [us-4.platform.sh] Charleston, United States (GCP)  [647 gC02eq/kWh]
> us-3.platform.sh

Default branch (--default-branch)
The default Git branch name for the project (the production environment)
Default: main
>

Are you sure you want to continue? [Y/n]

The Upsun bot will then provide an empty project for you and set the remote in your local git repository. Once done, you are ready to deploy:

Terminal

upsun push

The pipeline will build and deploy both apps. After a couple of minutes, the boilerplate apps will be available on: https://frontend.<branch>.<project>.<region>.platformsh.site https://app.<branch>.<project>.<region>.platformsh.site

You should be directed to the default pages for both the React and Laravel applications.

We can now connect to our remote database and cache services:

Terminal

upsun tunnel:open
Enter a number to choose an app:
  [0] app
  [1] frontend
 > 0

Are you sure you want to open SSH tunnel(s) to the environment main (type: production)? [Y/n]

SSH tunnel opened to database at: pgsql://main:main@127.0.0.1:30000/main
SSH tunnel opened to redis at: redis://127.0.0.1:30001

You can now edit your local watches/.env to use these services:

watches/.env

DB_CONNECTION=pgsql
DB_HOST=127.0.0.1
DB_PORT=3000
DB_DATABASE=main
DB_USERNAME=main
DB_PASSWORD=main

SESSION_DRIVER=redis
SESSION_LIFETIME=120
SESSION_ENCRYPT=false
SESSION_PATH=/
SESSION_DOMAIN=null

BROADCAST_CONNECTION=redis
FILESYSTEM_DISK=local
QUEUE_CONNECTION=redis

CACHE_STORE=redis
CACHE_PREFIX=

REDIS_CLIENT=phpredis
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=30001

You will need your local PHP runtime to have the phpredis extension loaded. Please refer to the following documentation on the Laravel site.

If you want to use test hostnames for your applications, edit /etc/hosts or your Windows host file with the following or any other names you like: 127.0.0.1 watches.test frontend.watches.test api.watches.test

If so, update the .env file with the proper hostname:

watches/.env

APP_URL=http://api.watches.test

We can now launch our development services. In two terminals, run:

Terminal

php artisan serve

Terminal

npm run dev -- --host=frontend.watches.test

Note the -- for the argument to be able to run. Your local applications are now available at http://frontend.watches.test:5137 and http://api.watches.test.

Getting some data about watches

As we will recommend a specific watch to our user, we must first have a database containing the potential results. Fortunately, Kaggle has a Luxury Watch Dataset available for download. Create an account if needed and download the CSV file.

Kaggle dataset

We will now move and rename the CSV file into our project:

Terminal

mkdir watches/seeds/
mv ~/Downloads/Luxury\ watch.csv ./seeds/watches-full.csv

Configuring our external services

For us to use OpenAI and Groq APIs, we need to inject our credentials. Create new API keys from their respective websites.

In Laravel, we will add the respective arrays in the services configuration:

config/services.php

<?php

return [

  'openai' => [
    'key' => env('OPENAI_API_KEY'),
  ],

  'groq' => [
    'key'=> env('GROQ_API_KEY'),
  ],

Once done, add the variables to your local .env file:

watches/.env

APP_URL=http://api.watches.test
[...]
OPENAI_API_KEY=sk-proj-**************
GROQ_API_KEY=gsk_**************

For them to work on Upsun, create two variables using the cli :

Terminal

upsun variable:create --name GROQ_API_KEY -l project --sensitive=true --prefix=env: --value=gsk_xxxxxxxxxxxxxx
upsun variable:create --name OPENAI_API_KEY -l project --sensitive=true --prefix=env: --value=sk-proj-xxxxxxxxxxxxxx

Let’s import our watches into the database

We will need a few things first. Create a Watch model. You can create it using php artisan make:model Watch --m or by adding the files manually:

watches/app/Models/Watch.php

<?php 

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
use Pgvector\Laravel\HasNeighbors;
use Pgvector\Laravel\Vector;

class Watch extends Model
{
    use HasFactory, HasNeighbors;

    protected $casts = ['embedding' => Vector::class];

    protected $fillable = [
        'brand',
        'model',
        'case_material',
        'strap_material',
        'movement_type',
        'water_resistance',
        'case_diameter_mm',
        'case_thickness_mm',
        'band_width_mm',
        'dial_color',
        'crystal_material',
        'complications',
        'power_reserve',
        'price_usd',
    ];
}

Note in the model that we use our Vector custom type to store and cast the embedding column. Create the migration file for the model:

Terminal

php artisan make migration create_watches_table

database/migrations/xxxx_xx_xx_xxxxxx_create_watches_table.php

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    /**
     * Run the migrations.
     */
    public function up(): void
    {
        Schema::create('watches', function (Blueprint $table) {
            $table->id();
            $table->string('brand');
            $table->string('model');
            $table->string('case_material')->nullable();
            $table->string('strap_material')->nullable();
            $table->string('movement_type')->nullable();
            $table->string('water_resistance')->nullable();
            $table->decimal('case_diameter_mm', 5, 2)->nullable();
            $table->decimal('case_thickness_mm', 5, 2)->nullable();
            $table->decimal('band_width_mm', 5, 2)->nullable();
            $table->string('dial_color')->nullable();
            $table->string('crystal_material')->nullable();
            $table->string('complications')->nullable();
            $table->string('power_reserve')->nullable();
            $table->integer('price_usd')->nullable();
            $table->vector('embedding', 1536)->nullable();
            $table->timestamps();
        });
    }

    /**
     * Reverse the migrations.
     */
    public function down(): void
    {
        Schema::dropIfExists('watches');
    }
};

Our embedding columns will use the vector type from the pg_vector package.

Migrate the database:

Terminal

php artisan migrate

We now need to code a job that will import the watches and generate the embedding. Create the command and the job:

Terminal

php artisan make:command Vectorize
php artisan make:job Vectorize

Our command is simple and only triggers the job:

watches/app/Console/Commands/Vectorize.php

<?php 

namespace App\Console\Commands;

use Illuminate\Console\Command;

class Vectorize extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'app:vectorize';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Vectorize the CSV file and stores it into the database.';

    /**
     * Execute the console command.
     */
    public function handle()
    {
        \App\Jobs\Vectorize::dispatchSync();
    }
}

Our Vectorize job is more complex. It will:

Parse the CSV and create the watches in the database
Combine all attributes of each watch into a text string that we will send to OpenAI to get an embedding
Store that embedding in our database as a vector

watches/app/Jobs/Vectorize.php

<?php

namespace App\Jobs;

use App\Models\Watch;
use DB;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use OpenAI;
use Pgvector\Laravel\Vector;

class Vectorize implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     */
    public function __construct()
    {
        //
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        // Empty DB
        DB::table('watches')->truncate();

        $client = OpenAI::client(config('services.openai.key'));

        // Parse the CSV and get the headers
        $file = base_path("seeds/watches-full.csv");
        $csv = array_map("str_getcsv", file($file));
        $header = array_shift($csv);

        // Transform the rows into an array with the header and the row
        $watches = array_map(fn($row) => array_combine($header, $row), $csv);

        // Loop the CSV and create the watches in database
        foreach($csv as $row) {
            try{
                Watch::create([
                    'brand' => $row[0],
                    'model' => $row[1],
                    'case_material' => $row[2],
                    'strap_material' => $row[3],
                    'movement_type' => $row[4],
                    'water_resistance' => $row[5],
                    'case_diameter_mm' => $row[6],
                    'case_thickness_mm' => $row[7],
                    'band_width_mm' => $row[8],
                    'dial_color' => $row[9],
                    'crystal_material' => $row[10],
                    'complications' => $row[11],
                    'power_reserve' => $row[12],
                    'price_usd' => preg_replace("/[^0-9.]/", "", $row[13]),
                ]);
            } catch(\Exception $e){
                ray($row);
            }
        }

        // Create a text version of the watch row that will list the 
        // attributes and their values as a large string
        $watchesInput = collect($watches)->map(function ($watch) {
            $str = '';
            foreach($watch as $key=>$item) {
                $str .= $key.':'.$item.',';
            }
            return rtrim($str, '|');
        });

        // Send the text version of all watches to the OpenAI embedding 
        // endpoint. We do a batch query to avoid sending 500 different API calls
        $response = $client->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $watchesInput->toArray()
        ]);

        // Loop through the embeddings sent and update our watches
        // in the database with the matching embedding
        foreach ($response->embeddings as $embedding) {
            $watch = Watch::find($embedding->index + 1);
            $watch->embedding = new Vector($embedding->embedding);
            $watch->save();
        }
    }
}

You can now run the job by triggering:

Terminal

php artisan app:vectorize

It should take a few seconds. Once done you can connect to your Postgres instance and check the results:

Terminal

upsun sql
Enter a number to choose an app:
  [0] app
  [1] frontend
 > 0

psql

psql (16.2 (Debian 16.2-1.pgdg120+2), server 15.6 (Debian 15.6-1.pgdg110+2))
Type "help" for help.

main=> SELECT brand, model FROM watches;

    brand        |          model
---------------------+--------------------------
 Omega               | Seamaster
 Tag Heuer           | Carrera
 Breitling           | Navitimer

[...]

Perfect, we have our database of 508 watches and the embeddings created from their content.

Create the API controller to get the search request

Let’s create the API controller for our frontend to query the database. The controller will take care of:

Parsing and validating the query value sent through the POST request
Creating an embedding of the query
Query the database using nearestNeighbors to find the most relevant watch
Send a streamed request to the Grok Chat API to create a proper answer with the details of the watch and the reason why it is the best.

As the response is streamed from Grok, we automatically stream it to our client using an Event Stream. This will allow us to display the answer being generated in real-time instead of waiting for the full answer being processes.

watches/app/Http/Controllers/WatchController.php

<?php 

namespace App\Http\Controllers;

use App\Helpers\MarkdownHelper;
use App\Http\Requests\SearchWatchRequest;
use App\Models\Watch;
use OpenAI;
use Pgvector\Laravel\Distance;
use Pgvector\Laravel\Vector;

class WatchController extends Controller
{
    public function search(SearchWatchRequest $request)
    {
        $embeddingClient = OpenAI::client(config('services.openai.key'));
        $chatClient = OpenAI::factory()
            ->withApiKey(config('services.groq.key'))
            ->withBaseUri('api.groq.com/openai/v1') // default: api.openai.com/v1
            ->make();

        // Create the embedding of the query
        $response = $embeddingClient->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $request->input('query')
        ]);

        $embedding = new Vector($response->embeddings[0]->embedding);

        // Find the closest watch
        $neighbor = Watch::query()->nearestNeighbors('embedding', $embedding, Distance::L2)->first();

        // Ask OpenAI for the closest watches
        $stream = $chatClient->chat()->createStreamed([
            'model' => 'llama3-8b-8192',
            'messages' => [
                [
                    'role' => 'system',
                    'content' => "You are an helpful AI that will recommend watches to the user. The watch selected is below with its attributes as a list. Your response is a markdown document.First, output the brand and model of the watch as a title.  Second, send the user a bullet list view of the watch attributes without doing any modification. Then, add a paragrah of why this watch matches their request using the watch attributes."
                ],
                [
                    'role' => 'user',
                    'content' => "The recommended watch is: " . MarkdownHelper::toMarkdown($neighbor)
                ],
                ['role' => 'user', 'content' => $request->input('query')],
            ],
        ]);

        return response()->stream(function () use ($stream) {
            $i = 0;
            $text = "";

            foreach($stream as $response){
                $text .= $response->choices[0]->delta->content;

                $i++;

                echo "event: chunk\n";
                echo "data: {\"index\": " . $i . ", \"chunk\": " . json_encode($response->choices[0]->delta->content) . "}\n\n";

                ob_flush();
                flush();

                // Break the loop if the client aborted the connection (closed the page)
                if (connection_aborted()) {
                    break;
                }
            }
        }, 200, [
            'Cache-Control' => 'no-cache',
            'X-Accel-Buffering' => 'no',
            'Content-Type' => 'text/event-stream',
        ]);
    }
}

As you can see, we are validating the POST request via the SearchWatchRequest validator. Remember to return true in the authorize() method, or you will get a 403 when POSTing to it.

watches/app/Http/Requests/SearchWatchRequest.php

<?php 

namespace App\Http\Requests;

use Illuminate\Foundation\Http\FormRequest;

class SearchWatchRequest extends FormRequest
{
    /**
     * Determine if the user is authorized to make this request.
     */
    public function authorize(): bool
    {
        return true;
    }

    /**
     * Get the validation rules that apply to the request.
     *
     * @return array<string, \Illuminate\Contracts\Validation\ValidationRule|array<mixed>|string>
     */
    public function rules(): array
    {
        return [
            'query' => 'required|string',
        ];
    }
}

In a production application, you must also validate that the user exists and is not abusing the system. You can leverage Laravel Sanctum for this.

Wrap up the Laravel configuration

Two tasks need to be done before our Laravel is fully operational. First, create a new route to handle the request:

watches/routes/api.php

<?php 

use Illuminate\Support\Facades\Route;

Route::post('/watch', [App\Http\Controllers\WatchController::class, 'search']);

Then, as we will query our API through a React frontend, we need to make CORS will be correctly set. Publish the CORS configuration file with:

Terminal

php artisan config:publish cors

Open cors.php and review the settings:

watches/config/cors.php

<?php 

return [
    'paths' => ['api/', 'api/watch'],
    'allowed_methods' => ['*'],
    'allowed_origins' => ['*'],
    'allowed_origins_patterns' => ['*'],
    'allowed_headers' => ['*'],
    'exposed_headers' => [],
    'max_age' => 0,
    'supports_credentials' => true,
];

Let’s test our API!

You can now use curl or a tool like Postman to query the API and get your result. As you can see, the result is streamed in chunks by our Laravel API. Congratulations!

API Test with Postman

Let’s deploy it!

Add the vectorize job in .upsun/config.yaml so it is done every time in the background after you deploy the app without impacting the app availability. You might want to trigger this manually if you have more data to avoid increasing your OpenAI bill too much.

.upsun/config.yaml

app:
    source:
      root: "watches"
    [...]
    hooks:
      build: |
        [...]        
      deploy: |
        [...]        
      post_deploy: |
        set -eux
        php artisan app:vectorize

Terminal

git add .
git commit -m "Working API"
upsun push

The deploy log will output the URLs for both endpoint. You can now retest the API with Postman on the Upsun API hostname.

Time to build our frontend

Let’s switch to our frontend React app now.

Terminal

cd ../frontend

As our UI will be really straightforward, we only need two basic shacdn/ui components:

Terminal

npx shadcn-ui@latest add button
npx shadcn-ui@latest add textarea

The components are now installed in src/components/ui.

We will use two different libraries to query our APIs:

axios would be used to query other potential endpoints of our Laravel API
@microsoft/fetch-event-source to stream the result from our API

Let’s install both:

Terminal

npm i axios
npm i @microsoft/fetch-event-source

# We also use markdown to format the output so let's install it
npm i react-markdown

We are now creating a shared axios instance for our project with the configuration and a method to define our API URL:

src/lib/axios.ts

/* src/lib/axios.ts */
import Axios from 'axios'

export const getApiURL = () => {
    if(window.location.hostname === 'watches.test') {
        return('http://api.watches.test')
    } else {
        return window.location.origin.replace('frontend', 'api')
    }
}

const entrypoint = getApiURL()

const axios = Axios.create({
    baseURL: entrypoint,
    headers: {
        'X-Requested-With': 'XMLHttpRequest',
        'Content-Type': 'application/json',
    },
})
export default axios

getApiURL will use our local development values if the hostname is watches.test. If not, it will replace the frontend string with api on any other hostname. This will allow us to make it functional in any preview environment on Upsun as well as production. You can always change the logic there to map your own deployment needs.

The main App.tsx

It’s time to build that frontend application layout and logic:

src/App.tsx

/* src/App.tsx */

import { useState } from 'react'
import './App.css'
import { Button } from './components/ui/button'
import { Textarea } from './components/ui/textarea'
import './index.css'
import Markdown from 'react-markdown'
import { fetchEventSource } from '@microsoft/fetch-event-source'
import { getApiURL } from './lib/axios'
function App() {

  const [prompt, setPrompt] = useState('')
  const [answer, setAnswer] = useState('Try queries like: "affordave dive watch", "dress watch with a black dial", "more affordable alternative to a Rolex submariner", ...')
  const [isStreaming, setIsStreaming] = useState(false)

  const submitForm = () => {
    newStreamResponse(prompt)
  }

  const newStreamResponse = async (prompt :string) => {
        const ctrl = new AbortController()
        setAnswer('')

        let messageContent = ''
        setIsStreaming(true)

        await fetchEventSource(`${getApiURL()}/api/watch`, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Accept': '*/*',
            },
            openWhenHidden: true,
            body: JSON.stringify({
              'query': prompt,
            }),

            onmessage(ev) {
                const tmp = JSON.parse(ev.data)

                if (tmp.chunk !== null) {
                    messageContent += tmp.chunk
                    setAnswer(messageContent)
                }
            },
            onclose() {
                // do not retry
                ctrl.abort()
            },
            onerror(err) {
                console.log(err)
                // toast.error('Something unexpected happened.', {
                //     position: toast.POSITION.BOTTOM_RIGHT,
                //     toastId: 'prompt-error',
                // })
                ctrl.abort()
                throw err;
            },
            signal: ctrl.signal,
        }).then(() => {
            setIsStreaming(false)
        })
    }

  return (
    <>
      <div className="grid w-full gap-2">
        <h1 className="text-4xl mb-8 text-center">⌚ Watch Finder</h1>
        <Textarea 
          onChange={(e) => setPrompt(e.target.value)} 
          placeholder="Describe the watch you are looking for..." 
          value={prompt} />
        <Button 
          onClick={() => submitForm()}
          disabled={isStreaming || prompt === ''}
          >Find a watch!</Button>

        <div className="p-4 rounded-sm my-8 bg-gray-100 prose lg:prose-md w-full max-w-max text-left">
          <Markdown>{answer}</Markdown>
        </div>
      </div>
    </>
  )
}

export default App

How it works:

The JSX is rendered on load. We use a <Markdown/> field to render the answer.
The user writes a prompt in the main <TextArea/> that is store in a variable prompt using useState.
When clicking the <Button/> , the script triggers submitForm() which creates a newStreamResponse
newStreamResponse starts by setting the state variable streaming as true so we can use it to freeze our UI while the request loads and reset the answer as well as creating a local variable named messageContent to store the stream content coming back from the API.
It then uses fetchEventSource to query our Laravel API. The openWhenHidden option specifies if the stream should be listened to when the tab or window is not in focus. We pass our prompt as the body of this POST request.
The onmessage method defines how we treat incoming data from the Event stream. Here we concatenate the new chunk to our messageContent and then update the state variable answer to show the result. In case your content is too long or the chunks are coming in too fast, you might want to delay the update to avoid too many renderings.
When the stream ends, we reset the streaming variable to false.

Final deploy

Once these files added, let’s deploy:

Terminal

git add .
git commit -m "Add frontend"
upsun push

The result

You can now use the frontend URL that Upsun gave you in the deploy log to access your app. Let’s query it!

Final Application Demo

Everything works as expected! You can see in my recording above, that I am using a custom hostname. In your Upsun console, click the Go Live button and follow the instructions to configure your DNS and setup the domain!

Setup domain

Conclusion

While this example is stripped down to the core, it is interesting to see how you can leverage embeddings for search, classification or any other Machine Learning process. With all the tooling now available, it becomes super accessible to everyone to query models and run these applications on a Cloud Applications Platform like Upsun.com.

Last updated on May 20, 2024