Build smarter with PHP — ship faster, learn deeper

Most Joomla sites serve the same content to every registered user regardless of what they have read before, what they searched for, or how long they spent on specific topics. A user who has read six articles about Joomla security gets the same homepage recommendations as someone who only reads performance tuning content. That is a missed opportunity every single time they log in.

Personalisation is not a new idea but it has historically required either expensive third-party platforms or a data science team to implement properly. What has changed is that OpenAI's embedding and completion APIs make it possible to build a genuinely useful personalisation engine inside your existing Joomla installation without either of those things.

What we are building here is a system that tracks what registered users click on, which articles they read, how long they spend reading, and what they search for. It uses that behaviour data to build an interest profile per user, then uses OpenAI embeddings to find articles that match that profile semantically, not just by category tag. The result is a recommendations module that gets more accurate the more a user engages with the site.

What you need: Joomla 4 or 5, PHP 8.1+, Composer, MySQL, and an OpenAI API key.

How the system works End to End

Before writing any code, the architecture is worth understanding clearly. There are three distinct parts to this system and keeping them separate makes the whole thing easier to build and maintain.

Part 1: Behaviour Tracking
        User reads an article, searches, or clicks a link
                ↓
        JavaScript sends event data to a Joomla plugin endpoint
                ↓
        Event stored in user_behaviour_events table

Part 2: Profile Building (runs on cron every hour)
        Fetch recent behaviour events per user
                ↓
        Build a text summary of user interests from event data
                ↓
        Send summary to OpenAI Embeddings API
                ↓
        Store interest profile vector in user_interest_profiles table

Part 3: Recommendations (runs on module render)
        Load current user's interest profile vector
                ↓
        Compare against pre-computed article embedding vectors
                ↓
        Return top N articles by cosine similarity
                ↓
        Render as a recommendations module on any page

The profile building step is the key insight here. Rather than trying to match individual behaviour events to articles directly, we build a text summary of each user's interests from their behaviour data, embed that summary as a vector, and then find articles that are semantically close to it. This means a user who reads articles about "Joomla template overrides" and "child theme development" will get recommendations about "Joomla layout XML" even if that exact phrase never appeared in their behaviour history.

Database Tables

We need three tables. One for raw behaviour events, one for computed user interest profiles, and one for pre-computed article embeddings. Run these in your Joomla database:

CREATE TABLE `#__user_behaviour_events` (
    `id`           INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
    `user_id`      INT UNSIGNED NOT NULL,
    `event_type`   ENUM('view','click','search','time_on_page') NOT NULL,
    `article_id`   INT UNSIGNED NULL,
    `search_query` VARCHAR(500) NULL,
    `duration_seconds` INT UNSIGNED NULL,
    `metadata`     JSON NULL,
    `created_at`   DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
    INDEX `idx_user_events` (`user_id`, `created_at`),
    INDEX `idx_article_events` (`article_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE `#__user_interest_profiles` (
    `user_id`      INT UNSIGNED PRIMARY KEY,
    `interest_summary` TEXT NOT NULL,
    `embedding`    JSON NOT NULL,
    `events_count` INT UNSIGNED NOT NULL DEFAULT 0,
    `updated_at`   DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE `#__article_embeddings` (
    `article_id`   INT UNSIGNED PRIMARY KEY,
    `embedding`    JSON NOT NULL,
    `indexed_at`   DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

MySQL does not have a native vector type so we are storing embeddings as JSON arrays. For sites with large article catalogues, over a few thousand articles, consider migrating to PostgreSQL with pgvector for proper vector indexing and faster similarity queries. For most Joomla sites MySQL with JSON embeddings works fine.

Part 1: Behaviour Tracking Plugin

Create a Joomla system plugin that handles two things: serving a lightweight JavaScript tracker, and receiving the behaviour events that tracker sends back.

Module structure at plugins/system/behaviourtracker/:

plugins/system/behaviourtracker/
    behaviourtracker.php
    behaviourtracker.xml
    src/
        Extension/
            BehaviourTracker.php
    media/
        js/
            tracker.js

The main plugin class at src/Extension/BehaviourTracker.php:

<?php

namespace Joomla\Plugin\System\BehaviourTracker\Extension;

use Joomla\CMS\Plugin\CMSPlugin;
use Joomla\CMS\Factory;
use Joomla\CMS\Uri\Uri;

class BehaviourTracker extends CMSPlugin
{
    public function onAfterDispatch(): void
    {
        $app  = Factory::getApplication();
        $user = Factory::getUser();

        // Only track logged-in users on site pages
        if ($user->guest || $app->isClient('administrator')) {
            return;
        }

        $input  = $app->getInput();
        $option = $input->getCmd('option');
        $view   = $input->getCmd('view');

        // Track article views automatically on the server side
        if ($option === 'com_content' && $view === 'article') {
            $articleId = $input->getInt('id');
            if ($articleId) {
                $this->recordEvent($user->id, 'view', $articleId);
            }
        }

        // Inject the JavaScript tracker on all frontend pages
        $doc     = $app->getDocument();
        $baseUrl = Uri::root();

        $doc->addScriptOptions('behaviourTracker', [
            'userId'   => $user->id,
            'endpoint' => $baseUrl . 'index.php?option=com_ajax&group=system&plugin=behaviourtracker&format=json',
            'articleId'=> ($option === 'com_content' && $view === 'article')
                          ? $input->getInt('id') : null,
        ]);

        $doc->addScript(Uri::root() . 'media/plg_system_behaviourtracker/js/tracker.js', [], ['defer' => true]);
    }

    public function onAjaxBehaviourtracker(): void
    {
        $app  = Factory::getApplication();
        $user = Factory::getUser();

        if ($user->guest) {
            echo json_encode(['status' => 'ignored']);
            $app->close();
        }

        $input     = $app->getInput()->json;
        $eventType = $input->getString('event_type');
        $articleId = $input->getInt('article_id', 0);
        $query     = $input->getString('search_query', '');
        $duration  = $input->getInt('duration_seconds', 0);

        $allowed = ['click', 'search', 'time_on_page'];

        if (!in_array($eventType, $allowed)) {
            echo json_encode(['status' => 'invalid']);
            $app->close();
        }

        $this->recordEvent(
            $user->id,
            $eventType,
            $articleId ?: null,
            $query ?: null,
            $duration ?: null
        );

        echo json_encode(['status' => 'ok']);
        $app->close();
    }

    private function recordEvent(
        int     $userId,
        string  $eventType,
        ?int    $articleId   = null,
        ?string $searchQuery = null,
        ?int    $duration    = null
    ): void {
        $db = Factory::getDbo();

        $row = (object)[
            'user_id'          => $userId,
            'event_type'       => $eventType,
            'article_id'       => $articleId,
            'search_query'     => $searchQuery,
            'duration_seconds' => $duration,
            'created_at'       => date('Y-m-d H:i:s'),
        ];

        $db->insertObject('#__user_behaviour_events', $row);
    }
}

Now the JavaScript tracker at media/js/tracker.js:

(function () {
    const config = Joomla.getOptions('behaviourTracker');

    if (!config || !config.userId) return;

    const endpoint = config.endpoint;
    const articleId = config.articleId;

    function send(payload) {
        navigator.sendBeacon
            ? navigator.sendBeacon(endpoint, JSON.stringify(payload))
            : fetch(endpoint, { method: 'POST', body: JSON.stringify(payload),
                                keepalive: true });
    }

    // Track time on page for article views
    if (articleId) {
        const startTime = Date.now();

        window.addEventListener('beforeunload', function () {
            const duration = Math.round((Date.now() - startTime) / 1000);

            // Only record if they spent more than 10 seconds
            if (duration > 10) {
                send({
                    event_type:       'time_on_page',
                    article_id:       articleId,
                    duration_seconds: duration,
                });
            }
        });
    }

    // Track internal link clicks to articles
    document.addEventListener('click', function (e) {
        const link = e.target.closest('a[href]');
        if (!link) return;

        const href = link.getAttribute('href');
        const match = href.match(/[?&]id=(\d+)/);

        if (match && href.includes('com_content')) {
            send({
                event_type: 'click',
                article_id: parseInt(match[1]),
            });
        }
    });

    // Track search queries
    const searchForm = document.querySelector('form[action*="com_search"], form[action*="com_finder"]');

    if (searchForm) {
        searchForm.addEventListener('submit', function () {
            const input = searchForm.querySelector('input[type="text"], input[name="q"], input[name="searchword"]');

            if (input && input.value.trim().length > 2) {
                send({
                    event_type:   'search',
                    search_query: input.value.trim(),
                });
            }
        });
    }
})();

Using navigator.sendBeacon for the time-on-page event is important. When a user navigates away, fetch requests get cancelled before they complete. sendBeacon is designed specifically for this situation and guarantees the request goes through even as the page unloads.

Part 2: Interest Profile Builder

This is the part that makes the recommendations smart rather than just "articles in the same category." Create this as a Joomla CLI task that runs on cron every hour.

Create components/com_personalisation/src/Service/ProfileBuilder.php:

<?php

namespace Joomla\Component\Personalisation\Site\Service;

use Joomla\CMS\Factory;
use OpenAI;

class ProfileBuilder
{
    private $openai;
    private int $lookbackDays = 30;
    private int $minEvents    = 3;

    public function __construct()
    {
        $params       = \JComponentHelper::getParams('com_personalisation');
        $this->openai = OpenAI::client($params->get('openai_api_key'));
    }

    public function buildForAllUsers(): array
    {
        $db = Factory::getDbo();

        // Find users who have behaviour events and need profile updates
        $query = $db->getQuery(true)
            ->select('DISTINCT e.user_id')
            ->from($db->quoteName('#__user_behaviour_events', 'e'))
            ->where($db->quoteName('e.created_at') . ' > ' .
                    $db->quote(date('Y-m-d H:i:s', strtotime("-{$this->lookbackDays} days"))))
            ->group($db->quoteName('e.user_id'))
            ->having('COUNT(*) >= ' . $this->minEvents);

        $userIds = $db->setQuery($query)->loadColumn();

        $built  = 0;
        $errors = 0;

        foreach ($userIds as $userId) {
            try {
                $this->buildForUser((int) $userId);
                $built++;
            } catch (\Exception $e) {
                $errors++;
                \JLog::add(
                    "Profile build failed for user {$userId}: " . $e->getMessage(),
                    \JLog::ERROR,
                    'com_personalisation'
                );
            }

            // Avoid hitting OpenAI rate limits
            usleep(100000);
        }

        return ['built' => $built, 'errors' => $errors];
    }

    public function buildForUser(int $userId): void
    {
        $db     = Factory::getDbo();
        $events = $this->fetchEvents($userId);

        if (empty($events)) {
            return;
        }

        // Build interest summary from behaviour events
        $summary = $this->buildInterestSummary($userId, $events);

        // Get embedding for the interest summary
        $response  = $this->openai->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $summary,
        ]);

        $embedding = $response->embeddings[0]->embedding;

        // Upsert the user interest profile
        $existing = $db->setQuery(
            $db->getQuery(true)
                ->select('user_id')
                ->from('#__user_interest_profiles')
                ->where('user_id = ' . (int) $userId)
        )->loadResult();

        $row = (object)[
            'user_id'          => $userId,
            'interest_summary' => $summary,
            'embedding'        => json_encode($embedding),
            'events_count'     => count($events),
            'updated_at'       => date('Y-m-d H:i:s'),
        ];

        $existing
            ? $db->updateObject('#__user_interest_profiles', $row, 'user_id')
            : $db->insertObject('#__user_interest_profiles', $row);
    }

    private function fetchEvents(int $userId): array
    {
        $db    = Factory::getDbo();
        $since = date('Y-m-d H:i:s', strtotime("-{$this->lookbackDays} days"));

        $query = $db->getQuery(true)
            ->select([
                'e.event_type',
                'e.article_id',
                'e.search_query',
                'e.duration_seconds',
                'a.title AS article_title',
                'a.catid',
                'c.title AS category_title',
            ])
            ->from($db->quoteName('#__user_behaviour_events', 'e'))
            ->leftJoin($db->quoteName('#__content', 'a') . ' ON a.id = e.article_id')
            ->leftJoin($db->quoteName('#__categories', 'c') . ' ON c.id = a.catid')
            ->where('e.user_id = ' . (int) $userId)
            ->where('e.created_at > ' . $db->quote($since))
            ->order('e.created_at DESC')
            ->setLimit(200);

        return $db->setQuery($query)->loadAssocList();
    }

    private function buildInterestSummary(int $userId, array $events): string
    {
        $viewedTitles    = [];
        $searchQueries   = [];
        $categories      = [];
        $longReads       = [];

        foreach ($events as $event) {
            if ($event['article_title']) {
                $viewedTitles[] = $event['article_title'];
            }

            if ($event['category_title']) {
                $categories[] = $event['category_title'];
            }

            if ($event['search_query']) {
                $searchQueries[] = $event['search_query'];
            }

            // Articles read for more than 90 seconds indicate strong interest
            if ($event['duration_seconds'] > 90 && $event['article_title']) {
                $longReads[] = $event['article_title'];
            }
        }

        // Deduplicate and limit to avoid overly long summaries
        $viewedTitles  = array_unique(array_slice($viewedTitles, 0, 20));
        $searchQueries = array_unique(array_slice($searchQueries, 0, 10));
        $categories    = array_values(array_unique($categories));
        $longReads     = array_unique($longReads);

        // Count category frequency to identify dominant interests
        $catCounts = array_count_values(array_column($events, 'category_title'));
        arsort($catCounts);
        $topCategories = array_keys(array_slice($catCounts, 0, 5, true));

        $parts = [];

        if (!empty($topCategories)) {
            $parts[] = "Primary interests: " . implode(', ', $topCategories) . ".";
        }

        if (!empty($searchQueries)) {
            $parts[] = "Searched for: " . implode(', ', $searchQueries) . ".";
        }

        if (!empty($longReads)) {
            $parts[] = "Read thoroughly: " . implode(', ', $longReads) . ".";
        }

        if (!empty($viewedTitles)) {
            $parts[] = "Also viewed: " . implode(', ', $viewedTitles) . ".";
        }

        return implode(' ', $parts);
    }
}

The interest summary construction is worth explaining. We weight long reads (over 90 seconds) separately because time spent reading is a stronger signal of genuine interest than a quick click. Category frequency tells us which topics dominate a user's browsing. Search queries tell us what they were actively looking for, not just passively browsing. Combining these three signals produces a richer interest summary than article titles alone.

Part 3: Article Indexer

Before we can recommend articles, we need embeddings for each article. This service indexes your Joomla content into the article embeddings table.

Create components/com_personalisation/src/Service/ArticleIndexer.php:

<?php

namespace Joomla\Component\Personalisation\Site\Service;

use Joomla\CMS\Factory;
use OpenAI;

class ArticleIndexer
{
    private $openai;
    private int $batchSize = 20;

    public function __construct()
    {
        $params       = \JComponentHelper::getParams('com_personalisation');
        $this->openai = OpenAI::client($params->get('openai_api_key'));
    }

    public function indexAll(bool $forceReindex = false): array
    {
        $db = Factory::getDbo();

        $query = $db->getQuery(true)
            ->select(['a.id', 'a.title', 'a.introtext', 'a.fulltext', 'c.title AS category'])
            ->from($db->quoteName('#__content', 'a'))
            ->leftJoin($db->quoteName('#__categories', 'c') . ' ON c.id = a.catid')
            ->where('a.state = 1');

        if (!$forceReindex) {
            // Only index articles without existing embeddings
            $query->leftJoin($db->quoteName('#__article_embeddings', 'ae') . ' ON ae.article_id = a.id')
                  ->where('ae.article_id IS NULL');
        }

        $articles = $db->setQuery($query)->loadObjectList();

        if (empty($articles)) {
            return ['indexed' => 0, 'skipped' => 0];
        }

        $indexed = 0;
        $errors  = 0;
        $batches = array_chunk($articles, $this->batchSize);

        foreach ($batches as $batch) {
            $texts = array_map(function ($article) {
                $body = strip_tags($article->introtext . ' ' . $article->fulltext);
                $body = preg_replace('/\s+/', ' ', $body);
                $body = substr(trim($body), 0, 2000);
                return $article->category . ': ' . $article->title . '. ' . $body;
            }, $batch);

            try {
                $response = $this->openai->embeddings()->create([
                    'model' => 'text-embedding-3-small',
                    'input' => $texts,
                ]);

                foreach ($batch as $i => $article) {
                    $embedding = $response->embeddings[$i]->embedding ?? null;

                    if (!$embedding) {
                        $errors++;
                        continue;
                    }

                    $this->upsertEmbedding($article->id, $embedding);
                    $indexed++;
                }

                usleep(200000);

            } catch (\Exception $e) {
                $errors += count($batch);
                \JLog::add('Article indexing error: ' . $e->getMessage(), \JLog::ERROR, 'com_personalisation');
            }
        }

        return ['indexed' => $indexed, 'errors' => $errors];
    }

    private function upsertEmbedding(int $articleId, array $embedding): void
    {
        $db = Factory::getDbo();

        $existing = $db->setQuery(
            $db->getQuery(true)
                ->select('article_id')
                ->from('#__article_embeddings')
                ->where('article_id = ' . $articleId)
        )->loadResult();

        $row = (object)[
            'article_id'  => $articleId,
            'embedding'   => json_encode($embedding),
            'indexed_at'  => date('Y-m-d H:i:s'),
        ];

        $existing
            ? $db->updateObject('#__article_embeddings', $row, 'article_id')
            : $db->insertObject('#__article_embeddings', $row);
    }
}

Notice the article text is prefixed with its category title before embedding. "Joomla Performance: Why your Joomla site is slow and how to fix it" produces a more accurate embedding than the title alone because the category provides context that helps distinguish articles on similar topics across different subject areas.

Part 4: The Recommendation Engine

Create components/com_personalisation/src/Service/RecommendationEngine.php:

<?php

namespace Joomla\Component\Personalisation\Site\Service;

use Joomla\CMS\Factory;

class RecommendationEngine
{
    private float $similarityThreshold = 0.70;

    public function getRecommendations(int $userId, int $limit = 5): array
    {
        $db = Factory::getDbo();

        // Load the user's interest profile
        $profile = $db->setQuery(
            $db->getQuery(true)
                ->select(['embedding', 'events_count'])
                ->from('#__user_interest_profiles')
                ->where('user_id = ' . $userId)
        )->loadObject();

        if (!$profile) {
            // No profile yet, return popular articles as fallback
            return $this->getPopularArticles($limit);
        }

        $userVector = json_decode($profile->embedding, true);

        if (empty($userVector)) {
            return $this->getPopularArticles($limit);
        }

        // Load all article embeddings
        $articles = $db->setQuery(
            $db->getQuery(true)
                ->select(['ae.article_id', 'ae.embedding', 'a.title', 'a.alias',
                           'a.catid', 'a.introtext', 'a.created', 'a.hits',
                           'c.title AS category', 'c.alias AS cat_alias'])
                ->from($db->quoteName('#__article_embeddings', 'ae'))
                ->join('INNER', $db->quoteName('#__content', 'a') . ' ON a.id = ae.article_id')
                ->join('LEFT', $db->quoteName('#__categories', 'c') . ' ON c.id = a.catid')
                ->where('a.state = 1')
        )->loadObjectList();

        // Get articles the user has already read so we don't recommend them again
        $readArticleIds = $db->setQuery(
            $db->getQuery(true)
                ->select('DISTINCT article_id')
                ->from('#__user_behaviour_events')
                ->where('user_id = ' . $userId)
                ->where('article_id IS NOT NULL')
                ->where("event_type IN ('view', 'click')")
        )->loadColumn();

        $readSet = array_flip($readArticleIds);

        // Score each article by cosine similarity to user interest vector
        $scored = [];

        foreach ($articles as $article) {
            // Skip already-read articles
            if (isset($readSet[$article->article_id])) {
                continue;
            }

            $articleVector = json_decode($article->embedding, true);

            if (empty($articleVector)) {
                continue;
            }

            $similarity = $this->cosineSimilarity($userVector, $articleVector);

            if ($similarity >= $this->similarityThreshold) {
                $scored[] = [
                    'article_id' => $article->article_id,
                    'title'      => $article->title,
                    'alias'      => $article->alias,
                    'category'   => $article->category,
                    'cat_alias'  => $article->cat_alias,
                    'introtext'  => strip_tags($article->introtext),
                    'created'    => $article->created,
                    'hits'       => $article->hits,
                    'similarity' => $similarity,
                ];
            }
        }

        // Sort by similarity score, highest first
        usort($scored, fn($a, $b) => $b['similarity'] <=> $a['similarity']);

        return array_slice($scored, 0, $limit);
    }

    private function cosineSimilarity(array $a, array $b): float
    {
        $dot  = 0.0;
        $magA = 0.0;
        $magB = 0.0;

        $len = min(count($a), count($b));

        for ($i = 0; $i < $len; $i++) {
            $dot  += $a[$i] * $b[$i];
            $magA += $a[$i] ** 2;
            $magB += $b[$i] ** 2;
        }

        $magA = sqrt($magA);
        $magB = sqrt($magB);

        return ($magA * $magB) > 0 ? $dot / ($magA * $magB) : 0.0;
    }

    private function getPopularArticles(int $limit): array
    {
        $db = Factory::getDbo();

        return $db->setQuery(
            $db->getQuery(true)
                ->select(['a.id AS article_id', 'a.title', 'a.alias',
                           'a.introtext', 'a.created', 'a.hits',
                           'c.title AS category', 'c.alias AS cat_alias'])
                ->from($db->quoteName('#__content', 'a'))
                ->leftJoin($db->quoteName('#__categories', 'c') . ' ON c.id = a.catid')
                ->where('a.state = 1')
                ->order('a.hits DESC')
                ->setLimit($limit)
        )->loadAssocList();
    }
}

The fallback to popular articles for users without a profile yet is important. A new registered user has no behaviour history, so returning nothing or an error is a bad experience. Popular articles are a reasonable default until enough behaviour data accumulates, which typically takes two to three sessions.

Excluding already-read articles from recommendations is something a lot of personalisation implementations skip. There is nothing more frustrating for a user than being recommended an article they read last week. The read set lookup adds minimal overhead and makes the recommendations feel genuinely useful.

Part 5: The Recommendations Module

Create a Joomla module that renders the recommendations anywhere on the site. Module structure at modules/mod_personalised_recommendations/:

modules/mod_personalised_recommendations/
    mod_personalised_recommendations.php
    mod_personalised_recommendations.xml
    tmpl/
        default.php

The main module file mod_personalised_recommendations.php:

<?php

defined('_JEXEC') or die;

use Joomla\CMS\Factory;
use Joomla\Component\Personalisation\Site\Service\RecommendationEngine;

$user = Factory::getUser();

if ($user->guest) {
    return;
}

$limit  = $params->get('article_count', 5);
$engine = new RecommendationEngine();

try {
    $recommendations = $engine->getRecommendations($user->id, $limit);
} catch (\Exception $e) {
    \JLog::add('Recommendations module error: ' . $e->getMessage(), \JLog::ERROR, 'mod_personalised_recommendations');
    $recommendations = [];
}

if (empty($recommendations)) {
    return;
}

require JModuleHelper::getLayoutPath('mod_personalised_recommendations', $params->get('layout', 'default'));

The module template at tmpl/default.php:

<?php defined('_JEXEC') or die; ?>

<div class="mod-personalised-recommendations">

    <h3><?php echo htmlspecialchars($params->get('header_text', 'Recommended for You')); ?></h3>

    <ul>
        <?php foreach ($recommendations as $item) : ?>
            <li>
                <a href="<?php echo JRoute::_(
                    'index.php?option=com_content&view=article&id=' . $item['article_id']
                    . '&catid=' . ($item['catid'] ?? '')
                ); ?>">
                    <?php echo htmlspecialchars($item['title']); ?>
                </a>

                <?php if ($params->get('show_category', 1) && !empty($item['category'])) : ?>
                    <span class="article-category">
                        <?php echo htmlspecialchars($item['category']); ?>
                    </span>
                <?php endif; ?>

                <?php if ($params->get('show_intro', 1) && !empty($item['introtext'])) : ?>
                    <p><?php echo htmlspecialchars(substr(strip_tags($item['introtext']), 0, 120)) . '...'; ?></p>
                <?php endif; ?>
            </li>
        <?php endforeach; ?>
    </ul>

</div>

Wiring It All Together With Cron

The profile builder needs to run on a schedule. Create a Joomla CLI script at cli/personalisation_cron.php:

<?php

define('_JEXEC', 1);
define('JPATH_BASE', dirname(__DIR__));

require_once JPATH_BASE . '/includes/defines.php';
require_once JPATH_BASE . '/includes/framework.php';

use Joomla\CMS\Factory;
use Joomla\Component\Personalisation\Site\Service\ProfileBuilder;
use Joomla\Component\Personalisation\Site\Service\ArticleIndexer;

$app = Factory::getApplication('cli');

$task = $argv[1] ?? 'profiles';

switch ($task) {
    case 'index':
        $indexer = new ArticleIndexer();
        $result  = $indexer->indexAll();
        echo "Indexed: {$result['indexed']}, Errors: {$result['errors']}\n";
        break;

    case 'profiles':
    default:
        $builder = new ProfileBuilder();
        $result  = $builder->buildForAllUsers();
        echo "Profiles built: {$result['built']}, Errors: {$result['errors']}\n";
        break;
}

Add these to your server crontab:

# Rebuild user interest profiles every hour
0 * * * * php /path/to/joomla/cli/personalisation_cron.php profiles >> /var/log/joomla_personalisation.log 2>&1

# Index new articles every 6 hours
0 */6 * * * php /path/to/joomla/cli/personalisation_cron.php index >> /var/log/joomla_personalisation.log 2>&1

Run the article indexer manually first before anything else:

php /path/to/joomla/cli/personalisation_cron.php index

This builds embeddings for all your existing articles. Depending on how many articles you have, this might take a few minutes and cost a small amount in OpenAI API tokens. For 500 articles it typically costs less than $0.10 using text-embedding-3-small.

What the Recommendations Look Like in Practice

Here is a realistic example of what the system produces after a user has been active for a few sessions. The user has read articles about Joomla template development, searched for "override layout XML", and spent over two minutes reading an article about Joomla child templates.

Their interest summary built by the profile builder:

Primary interests: Joomla Development, Joomla Theming.
Searched for: override layout XML, Joomla template child.
Read thoroughly: How to Create a Child Template in Joomla 5.
Also viewed: Joomla Template Overrides Explained, Understanding Joomla Layout XML,
             Joomla Module Chrome Types, Adding Custom CSS to a Joomla Template.

Top recommendations returned by the engine (articles they have not read yet):

[
  {
    "title": "Joomla 5 Template Positions: A Complete Guide",
    "category": "Joomla Development",
    "similarity": 0.912
  },
  {
    "title": "How to Override Joomla Core Templates Without Hacking Core",
    "category": "Joomla Development",
    "similarity": 0.887
  },
  {
    "title": "Using Bootstrap 5 Effectively in Custom Joomla Templates",
    "category": "Joomla Theming",
    "similarity": 0.871
  },
  {
    "title": "Joomla Module Assignment by Menu Item and User Group",
    "category": "Joomla Development",
    "similarity": 0.843
  },
  {
    "title": "Debugging Joomla Layout Issues with the Developer Toolbar",
    "category": "Joomla Development",
    "similarity": 0.831
  }
]

All five are relevant, none are articles the user has already read, and the recommendations are semantically accurate even though the user never used the exact phrase "template positions" or "Bootstrap" in their search queries. That is the embedding similarity working correctly.

A Few Things Worth Knowing Before You Deploy

The cosine similarity calculation in PHP is fine for article catalogues up to a few thousand articles. Once you get beyond that, the in-memory comparison of every article embedding against every user profile starts to add up. At that scale, moving the article embeddings to PostgreSQL with pgvector and running the similarity query in the database will keep response times fast.

Privacy is worth thinking about carefully before deploying behaviour tracking. Depending on your jurisdiction, tracking logged-in user behaviour may require disclosure in your privacy policy and potentially explicit consent. At minimum, update your privacy policy to mention that behaviour data is collected to improve content recommendations. For sites with European users, review GDPR requirements around behavioural profiling before going live.

The 30-day lookback window in the profile builder is a reasonable default but worth tuning for your site. On a site where users visit daily, 30 days captures good signal. On a site where users visit monthly, a 90-day window gives more data to work with. Adjust the $lookbackDays property in ProfileBuilder to match your site's typical visit cadence.

Finally, add a simple feedback mechanism if you can. Even a small thumbs up or thumbs down on recommended articles gives you data to validate whether the recommendations are actually landing well with your users. If most users ignore or actively dismiss recommendations, that tells you something about either the similarity threshold, the quality of your interest summaries, or the article content itself. Without that feedback loop, you are flying blind on whether the engine is actually helping.

Magento 2 has a reputation for being complicated, and honestly that reputation is earned. The learning curve is steep, the documentation has gaps, and some of its architectural decisions only make sense once you have been burned by the alternative. I have been working with Magento 2 since its early releases and the tips in this post come directly from real projects, real mistakes, and things I wish someone had told me before I spent hours figuring them out the hard way.

This is not a list of things you will find in the official documentation. These are the practical things that make daily Magento 2 development faster, less frustrating, and more maintainable.

Development Workflow Tips

1. Always work in developer mode during development

This sounds obvious but I have seen teams develop in default mode and wonder why their template changes are not showing up. In developer mode, Magento disables most caching, enables full error reporting, and processes frontend assets on the fly. Switch modes from the command line:

php bin/magento deploy:mode:set developer

The flip side is that production mode compiles and minifies everything, which is why you must run the full deployment commands before pushing to production. Never deploy to production in developer mode. It is slower and exposes internal errors to visitors.

2. Know your cache types and clear only what you need

Running php bin/magento cache:flush clears everything and takes time. In most cases during development you only need to clear specific cache types. Clearing just the config cache after a config change, or just the layout cache after a layout XML change, is significantly faster.

# Clear only config cache
php bin/magento cache:clean config

# Clear layout and block cache after layout changes
php bin/magento cache:clean layout block_html

# Clear full page cache after template changes
php bin/magento cache:clean full_page

# See all cache types and their status
php bin/magento cache:status

Learn which cache type corresponds to which kind of change. It will save you minutes every time you test something.

3. Use n98-magerun2 for everything the CLI does not cover

n98-magerun2 is the Swiss Army knife for Magento 2 development. It extends Magento's CLI with hundreds of useful commands for database management, admin user creation, module management, and much more.

# Install globally
composer global require n98/magerun2

# Create an admin user without going through the UI
n98-magerun2 admin:user:create --username=devadmin --email=dev@example.com \
    --password=Dev@12345 --firstname=Dev --lastname=Admin

# Run a database query directly
n98-magerun2 db:query "SELECT * FROM admin_user"

# Open a MySQL console connected to Magento's database
n98-magerun2 db:console

# Show all configuration values for a path
n98-magerun2 config:store:get catalog/search/*

The db:console command alone is worth installing it for. No more hunting for database credentials in env.php every time you need to run a query.

4. Setup Xdebug properly

Too many Magento developers rely on var_dump and log files for debugging. Xdebug with step-through debugging changes how you work entirely. You can inspect the full call stack, watch variable values change in real time, and understand exactly what Magento is doing at any point in its request cycle.

The most common Xdebug configuration issue with Magento is the request timeout. Set a generous timeout in your php.ini because Magento's bootstrap takes longer than a typical PHP application and Xdebug adds overhead:

xdebug.mode=debug
xdebug.start_with_request=yes
xdebug.client_host=host.docker.internal  ; if using Docker
xdebug.client_port=9003
xdebug.max_execution_time=0  ; disable timeout during debug sessions

5. Use the Magento 2 console to run cron manually

Waiting for Magento's cron to run on its schedule during development is painful. You can trigger specific cron groups manually:

# Run all cron jobs
php bin/magento cron:run

# Run a specific cron group
php bin/magento cron:run --group=index

# Check cron schedule
php bin/magento cron:schedule

If something is not updating after you expect it to, a manual cron run is often the answer.

Performance Tips

6. Enable Redis for Cache and Session Storage

File-based cache storage is fine for development but it will not scale in production. Redis handles Magento's cache and session storage significantly better, especially under concurrent load. Configure it in app/etc/env.php:

'cache' => [
    'frontend' => [
        'default' => [
            'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
            'backend_options' => [
                'server'   => '127.0.0.1',
                'port'     => '6379',
                'database' => '0',
                'password' => '',
            ],
        ],
        'page_cache' => [
            'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
            'backend_options' => [
                'server'   => '127.0.0.1',
                'port'     => '6379',
                'database' => '1',  // separate database for full page cache
            ],
        ],
    ],
],
'session' => [
    'save'         => 'redis',
    'redis' => [
        'host'     => '127.0.0.1',
        'port'     => '6379',
        'database' => '2',
    ],
],

Use a separate Redis database number for each type. Mixing the page cache and session data in the same database means a cache flush will wipe active sessions, which logs out all your customers. Learned that one the hard way on a live site.

7. Flat Catalog tables speed up large catalogues

By default Magento stores product and category attributes using the EAV (Entity-Attribute-Value) model. For catalogues with thousands of products and many attributes, this means dozens of JOIN operations per product list query. Flat tables consolidate all attributes into a single table per entity type, which is dramatically faster for reads.

# Enable flat catalog in admin: Stores > Configuration > Catalog > Catalog
# Then rebuild the flat tables
php bin/magento indexer:reindex catalog_category_flat
php bin/magento indexer:reindex catalog_product_flat

The tradeoff is that the flat tables need to be kept in sync with the EAV tables, which adds a small overhead to product saves. For read-heavy stores with large catalogues this is almost always worth it.

8. Optimise your indexers and set them to update on schedule

Magento has multiple indexers that rebuild data structures when products, categories, or prices change. Running them on "Update on Save" means every product edit triggers a full or partial reindex, which blocks the save operation and slows down the admin. On stores with large catalogues this can make product editing painfully slow.

# Set all indexers to update on schedule
php bin/magento indexer:set-mode schedule

# Or set a specific indexer
php bin/magento indexer:set-mode schedule catalogrule_rule

# Check current indexer status
php bin/magento indexer:status

With schedule mode, reindexing happens on cron runs rather than during save operations. Editors get fast saves, the index stays reasonably fresh, and you avoid timeout issues when making bulk changes.

9. Enable Varnish for full page cache in production

Magento's built-in full page cache is decent but Varnish is significantly faster for high-traffic stores. Magento generates a Varnish configuration file for you:

php bin/magento varnish:vcl:generate --export-version=6 > /etc/varnish/default.vcl

Set the caching application to Varnish in the admin under Stores, Configuration, Advanced, System, Full Page Cache. One thing that catches developers out: after enabling Varnish you need to set the HTTP port to 80 and let Varnish handle requests on that port, with Magento behind it on a different port. Your web server config needs to reflect this or you will get redirect loops.

10. Use Asynchronous and deferred indexing for imports

If you run regular product imports via CSV or API, synchronous indexing after each import will slow things to a crawl. Enable asynchronous reindexing and let the cron handle it:

# Enable async grid indexing
php bin/magento config:set dev/grid/async_indexing 1

For bulk imports using the Import API, make sure you call php bin/magento indexer:reindex after the import completes rather than relying on the save events to trigger reindexing. Batch imports at the end, not during.

Backend and Admin Tips

11. Customise Admin Grids Without Rewriting the Whole Grid

One of the most common admin customisations is adding a column to an existing grid, like the orders grid or the customers grid. The wrong way is to override the entire grid PHP class. The right way is to use a UI Component XML file to extend just what you need.

Create a layout XML file in your module at view/adminhtml/layout/sales_order_grid.xml:

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <body>
        <referenceBlock name="sales_order_grid">
            <arguments>
                <argument name="data" xsi:type="array">
                    <item name="config" xsi:type="array">
                        <item name="update_url" xsi:type="url" path="mui/index/render"/>
                    </item>
                </argument>
            </arguments>
        </referenceBlock>
    </body>
</page>

Then extend the UI component definition in view/adminhtml/ui_component/sales_order_grid.xml to add your column. This approach survives Magento upgrades far better than class overrides because you are extending, not replacing.

12. Use Virtual types to avoid unnecessary classes

Virtual types are one of Magento's most underused features. They let you create a variation of an existing class with different constructor arguments, without writing a new PHP file. This is useful when you need multiple instances of the same class configured differently.

<!-- di.xml -->
<virtualType name="Vendor\Module\Model\CustomLogger"
             type="Magento\Framework\Logger\Monolog">
    <arguments>
        <argument name="name" xsi:type="string">customModule</argument>
        <argument name="handlers" xsi:type="array">
            <item name="system" xsi:type="object">
                Vendor\Module\Logger\Handler\Custom
            </item>
        </argument>
    </arguments>
</virtualType>

<!-- Use the virtual type as a dependency -->
<type name="Vendor\Module\Model\SomeService">
    <arguments>
        <argument name="logger" xsi:type="object">
            Vendor\Module\Model\CustomLogger
        </argument>
    </arguments>
</type>

This creates a logger instance with custom configuration without writing a new Logger class. Clean, upgrade-safe, and takes five minutes once you understand the pattern.

13. Use Plugins (Interceptors) Instead of Rewrites

Magento 1 developers often reach for class rewrites out of habit. In Magento 2, plugins are almost always the better choice. Plugins let you execute code before, after, or around a public method of any class without replacing the entire class.

<!-- di.xml -->
<type name="Magento\Catalog\Model\Product">
    <plugin name="vendor_module_product_plugin"
            type="Vendor\Module\Plugin\ProductPlugin"
            sortOrder="10"
            disabled="false"/>
</type>

<?php

namespace Vendor\Module\Plugin;

use Magento\Catalog\Model\Product;

class ProductPlugin
{
    // Runs after getName() and can modify the return value
    public function afterGetName(Product $subject, string $result): string
    {
        if ($subject->getTypeId() === 'bundle') {
            return $result . ' (Bundle)';
        }

        return $result;
    }

    // Runs before setPrice() and can modify the arguments
    public function beforeSetPrice(Product $subject, float $price): array
    {
        // Ensure price is never negative
        return [max(0, $price)];
    }
}

Multiple plugins can target the same method and the sortOrder attribute controls execution order. This means your customisation coexists with other modules' plugins rather than one rewrite blocking another entirely.

14. Log to custom files, not the default system log

Everything in Magento logs to var/log/system.log by default. On an active store this file grows fast and becomes almost useless for finding specific module issues. Create a custom logger for your module that writes to its own file:

<?php

namespace Vendor\Module\Logger;

use Monolog\Logger;

class Handler extends \Magento\Framework\Logger\Handler\Base
{
    protected $loggerType = Logger::DEBUG;
    protected $fileName   = '/var/log/vendor_module.log';
}

Register this handler in your module's di.xml using a virtual type (see tip 12 above) and inject it wherever you need logging. Your module's debug output goes to its own file and does not get buried in the system log noise.

Frontend and Theming Tips

15. Extend Parent Themes Rather Than Modifying Them Directly

Never modify Luma or Blank theme files directly. Magento updates will overwrite your changes and you will have no way of knowing what was changed. Always create a child theme that declares the parent:

<!-- app/design/frontend/Vendor/mytheme/theme.xml -->
<theme xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:noNamespaceSchemaLocation="urn:magento:framework:Config/etc/theme.xsd">
    <title>Vendor My Theme</title>
    <parent>Magento/luma</parent>
</theme>

To override a template, copy it from the parent theme into the same relative path in your theme and edit the copy. Magento's theme fallback system will use your version. To override a layout, create a layout XML file with the same name in your theme's layout directory and add only the changes you need.

16. Use the less variable override system properly

Magento 2 uses Less for stylesheets and has a variable system that lets you customise colours, fonts, and spacing without touching the source Less files. Override variables in your theme by creating web/css/source/_variables.less:

// Override the primary colour across the entire theme
@color-orange: #e84040;

// Change the default font
@font-family__base: 'Inter', sans-serif;
@font-family__serif: Georgia, serif;

// Button styling
@button__background: @color-orange;
@button__border-radius: 4px;

// Navigation
@navigation__background: #1a1a2e;
@navigation-level0-item__color: #ffffff;

This propagates through Magento's entire Less stack without you touching a single source file. Most visual customisations can be done entirely through variable overrides. Only write custom Less for things the variable system does not cover.

17. Understand static content deployment

The most common frontend issue I see in production deployments is forgetting to deploy static content, or deploying it for the wrong locales.

# Deploy for all themes and locales
php bin/magento setup:static-content:deploy

# Deploy for specific locales only (faster)
php bin/magento setup:static-content:deploy en_US en_GB

# Deploy a specific theme
php bin/magento setup:static-content:deploy -t Vendor/mytheme en_US

# Use parallel deployment for speed on large stores
php bin/magento setup:static-content:deploy --jobs=4 en_US

If your static content looks correct in developer mode but breaks in production, you almost certainly have a deployment issue. Check that all your locales are included in the deploy command and that the deployment ran successfully after the last code change.

18. Use RequireJS config to load JavaScript properly

Adding JavaScript in Magento 2 is not as simple as dropping a script tag in a template. The right way is through RequireJS, which manages dependencies and load order. Define your JavaScript modules in view/frontend/requirejs-config.js:

var config = {
    map: {
        '*': {
            'customSlider': 'Vendor_Module/js/custom-slider',
        }
    },
    shim: {
        'customSlider': {
            deps: ['jquery'],
        }
    },
    paths: {
        'slick': 'Vendor_Module/js/vendor/slick.min',
    }
};

Then use it in a template or another JavaScript file:

require(['customSlider', 'jquery'], function(CustomSlider, $) {
    var slider = new CustomSlider({
        element: $('.product-gallery'),
        autoplay: true,
    });
});

The RequireJS approach means your JavaScript is only loaded when needed, dependencies are guaranteed to be available, and your code does not conflict with other modules' JavaScript.

Common Mistakes to avoid

19. Do Not use ObjectManager directly in your code

You will see ObjectManager used in old tutorials and even in some core Magento code. Do not follow that pattern in your own modules. Using ObjectManager directly bypasses dependency injection, makes code untestable, and creates hidden dependencies.

<?php

// Wrong. Do not do this.
$objectManager = \Magento\Framework\App\ObjectManager::getInstance();
$product = $objectManager->create(\Magento\Catalog\Model\Product::class);

// Right. Inject dependencies through the constructor.
class MyService
{
    public function __construct(
        private \Magento\Catalog\Model\ProductFactory $productFactory
    ) {}

    public function doSomething(): void
    {
        $product = $this->productFactory->create();
    }
}

The only acceptable place to use ObjectManager directly is in factory classes and proxy classes, and even there Magento generates those automatically through its code generation system.

20. Do Not modify core files

Every Magento update will overwrite your changes. Use plugins, preferences, layout overrides, and template overrides instead. If you find yourself editing a file in vendor/magento/, stop and find the proper extension point instead. There is always one.

21. Run Code egneration after adding new classes

Magento generates proxy classes, factory classes, and interceptors automatically. When you add a new class that uses dependency injection, you may need to regenerate this code:

# Generate interceptors and factories
php bin/magento setup:di:compile

# Clear generated code if something seems wrong
rm -rf generated/code/*
php bin/magento setup:di:compile

If you are getting "Class does not exist" errors for factory or proxy classes, this is almost always the fix. In developer mode Magento generates these on the fly, but in production mode they must be compiled ahead of time.

22. Check your module's sequence in module.xml

If your module depends on another module being loaded first, declare that dependency in etc/module.xml. Without it, your di.xml or layout XML might be processed before the module it depends on, causing hard-to-debug errors.

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="Vendor_Module" setup_version="1.0.0">
        <sequence>
            <module name="Magento_Catalog"/>
            <module name="Magento_Sales"/>
        </sequence>
    </module>
</config>

One last thing worth mentioning

Magento 2 rewards developers who take the time to understand its architecture rather than fighting against it. The dependency injection system, the plugin architecture, the UI component framework, and the layout XML system all have learning curves but they exist for good reasons. Once you work with them properly instead of around them, your code is more maintainable, survives upgrades better, and plays well with other modules.

The most expensive Magento projects I have seen are the ones where developers skipped the architecture and went straight to hacking. The technical debt accumulates fast and the upgrade path becomes a rewrite. The extra time spent doing things the Magento way upfront is always worth it.

If there is a specific Magento 2 topic you want covered in more depth, drop a comment below.

Every Laravel project that needs AI ends up with a different implementation. One project uses the OpenAI PHP client directly. Another one uses a wrapper someone wrote three years ago that is no longer maintained. A third one is tightly coupled to a specific model, so switching from GPT-4o to Claude requires rewriting half the service layer.

Prism PHP solves this properly. It is a Laravel package that gives you a single, consistent API for working with multiple AI providers. OpenAI, Anthropic Claude, Ollama for local models, Mistral, Gemini, and more, all through the same fluent interface. You switch providers by changing one line. Your application code does not care which model is behind it.

This post covers the full picture. All the supported providers and when to use each one, text generation with structured output, tool calling so your AI can actually interact with your application, and embeddings for semantic search. I will tie all three together with a real-world example at the end so you can see how they work as a system rather than isolated features.

What you need:

• Laravel 10 or 11
• PHP 8.1+
• Composer
• API keys for whichever providers you plan to use.
• Ollama requires a local install but is free to run.

Why Prism instead of the OpenAI Client directly

The openai-php/laravel client is solid and I have used it in several projects on this blog. But it locks you into OpenAI. If you want to try Claude for a specific task, or use a local Ollama model for development to avoid API costs, you are writing separate integration code for each one.

Prism is inspired by the Vercel AI SDK, which solved this same problem in the JavaScript world. The idea is simple: define a unified interface, write drivers for each provider, and let application code stay completely provider-agnostic. The practical benefits are real.

You can use GPT-4o for general generation, Claude for tasks where it performs better (long document analysis, nuanced writing), and a local Ollama model during development so you are not burning API credits on every test run. All through the same application code. That flexibility is genuinely useful once you start building production AI features.

Supported Providers

Prism currently ships with first-party support for these providers. Each has its own strengths and the right choice depends on the task.

Provider	Best For	Key Models	Cost
OpenAI	General generation, embeddings, function calling	GPT-4o, GPT-4o-mini, text-embedding-3-small	Pay per token
Anthropic	Long documents, reasoning, nuanced analysis	Claude 3.7 Sonnet, Claude 3.5 Haiku	Pay per token
Ollama	Local development, privacy-sensitive data, zero API cost	Llama 3, Mistral, Phi-3, any Ollama model	Free (runs locally)
Mistral	Efficient generation, European data residency	Mistral Large, Mistral Small	Pay per token
Google Gemini	Multimodal tasks, audio and video input	Gemini 1.5 Flash, Gemini 1.5 Pro	Pay per token
xAI (Grok)	Real-time data awareness, alternative to GPT-4	Grok-2	Pay per token

My general approach: OpenAI for embeddings and general tasks, Claude for anything involving long content or nuanced judgment, Ollama for local development. That combination covers most application needs while keeping costs reasonable.

Installation and Configuration

composer require prism-php/prism

Publish the config file:

php artisan vendor:publish --tag=prism-config

This generates config/prism.php. Add your provider credentials to .env:

# OpenAI
OPENAI_API_KEY=sk-your-openai-key

# Anthropic
ANTHROPIC_API_KEY=sk-ant-your-anthropic-key

# Mistral
MISTRAL_API_KEY=your-mistral-key

# Google Gemini
GEMINI_API_KEY=your-gemini-key

# xAI
XAI_API_KEY=your-xai-key

# Ollama runs locally, no API key needed
# Default URL is http://localhost:11434

The config/prism.php file maps these to the relevant providers. You can also set default models per provider here, which saves repeating the model name in every call.

Part 1: Text Generation and Structured Output

The core feature and the one you will use most. Prism's text generation API is chainable and reads naturally, which is one of the things that makes it feel like a proper Laravel package rather than a thin wrapper.

Basic Text Generation

Here is the same prompt sent to three different providers, with zero application code changes between them:

<?php

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

// OpenAI
$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withSystemPrompt('You are a helpful PHP development assistant.')
    ->withPrompt('Explain what a service container is in Laravel.')
    ->asText();

echo $response->text;

// Swap to Claude, same code
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-7-sonnet-latest')
    ->withSystemPrompt('You are a helpful PHP development assistant.')
    ->withPrompt('Explain what a service container is in Laravel.')
    ->asText();

echo $response->text;

// Or run it locally with Ollama during development
$response = Prism::text()
    ->using(Provider::Ollama, 'llama3')
    ->withSystemPrompt('You are a helpful PHP development assistant.')
    ->withPrompt('Explain what a service container is in Laravel.')
    ->asText();

echo $response->text;

That is the core value proposition right there. Same interface, different provider. If OpenAI has an outage or you want to A/B test Claude versus GPT-4o on a specific prompt, you change one line.

Structured Output with Schema Validation

Getting raw text back is fine for simple tasks. For anything that feeds into application logic, you want structured output. Prism handles this through schema definitions that map directly to PHP objects.

<?php

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\IntegerSchema;
use Prism\Prism\Schema\ArraySchema;

$schema = new ObjectSchema(
    name: 'article_analysis',
    description: 'Analysis of a PHP tutorial article',
    properties: [
        new StringSchema('summary', 'One sentence summary of the article'),
        new IntegerSchema('difficulty_level', 'Difficulty from 1 (beginner) to 5 (expert)'),
        new StringSchema('primary_topic', 'The main topic of the article'),
        new ArraySchema(
            'key_concepts',
            'Key technical concepts covered',
            new StringSchema('concept', 'A technical concept mentioned in the article')
        ),
        new ArraySchema(
            'prerequisite_knowledge',
            'What the reader should know before reading this',
            new StringSchema('prerequisite', 'A prerequisite concept or skill')
        ),
    ],
    requiredFields: ['summary', 'difficulty_level', 'primary_topic', 'key_concepts']
);

$articleContent = "Your article text goes here...";

$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withSystemPrompt('You analyse PHP and Laravel tutorial articles.')
    ->withPrompt("Analyse this article:\n\n{$articleContent}")
    ->withSchema($schema)
    ->asStructured();

// $response->structured is a fully typed PHP array matching your schema
$analysis = $response->structured;

echo $analysis['summary'];
echo $analysis['difficulty_level'];
echo implode(', ', $analysis['key_concepts']);

No more parsing freeform text. No more stripping markdown fences from JSON responses. Prism handles the structured output negotiation with the model and gives you a validated PHP array. If the model returns something that does not match the schema, Prism throws rather than silently returning garbage data.

Multi-Turn Conversations

<?php

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\AssistantMessage;

$history = [
    new UserMessage('What is the difference between Laravel jobs and events?'),
    new AssistantMessage('Jobs are queued tasks for deferred work. Events are for broadcasting that something happened in your application...'),
    new UserMessage('Can you show me a code example of a job?'),
];

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-7-sonnet-latest')
    ->withSystemPrompt('You are a Laravel expert.')
    ->withMessages($history)
    ->asText();

echo $response->text;

Part 2: Tool Calling

Tool calling is where things get genuinely interesting. Instead of the AI just generating text, you give it tools it can call, functions that interact with your actual application. The model decides when to use a tool based on the user's request, calls it, gets the result, and incorporates it into its response.

Without tool calling, an AI assistant can only work with what it was trained on. With tool calling, it can check live database records, call external APIs, perform calculations, and do anything else you give it a tool for.

Defining a Tool

<?php

use Prism\Prism\Tool;

// A tool that looks up an order from your database
$orderLookupTool = Tool::as('get_order_status')
    ->for('Look up the status and details of a customer order by order number')
    ->withStringParameter('order_number', 'The order number to look up, e.g. ORD-2025-1234')
    ->using(function (string $order_number): string {
        $order = \App\Models\Order::where('order_number', $order_number)
            ->with('items')
            ->first();

        if (!$order) {
            return json_encode(['error' => 'Order not found']);
        }

        return json_encode([
            'order_number' => $order->order_number,
            'status'       => $order->status,
            'placed_at'    => $order->created_at->format('d M Y'),
            'total'        => '$' . number_format($order->total, 2),
            'items'        => $order->items->count(),
            'tracking'     => $order->tracking_number ?? 'Not yet assigned',
        ]);
    });

The model sees the tool name, description, and parameter definitions. When a user asks something like "what is the status of my order ORD-2025-4821", the model recognises it needs the order lookup tool, calls it with the order number extracted from the message, gets the JSON result back, and uses it to form a natural language response.

Using Multiple Tools Together

<?php

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Tool;

$orderLookupTool = Tool::as('get_order_status')
    ->for('Look up order status by order number')
    ->withStringParameter('order_number', 'The order number')
    ->using(function (string $order_number): string {
        // Your order lookup logic here
        return json_encode(['status' => 'shipped', 'tracking' => 'TRK123456']);
    });

$productSearchTool = Tool::as('search_products')
    ->for('Search the product catalogue by keyword')
    ->withStringParameter('keyword', 'Search term to find products')
    ->withIntegerParameter('limit', 'Maximum number of results to return')
    ->using(function (string $keyword, int $limit = 5): string {
        $products = \App\Models\Product::search($keyword)
            ->take($limit)
            ->get(['name', 'price', 'in_stock']);

        return json_encode($products->toArray());
    });

$refundPolicyTool = Tool::as('get_refund_policy')
    ->for('Retrieve the current refund and returns policy')
    ->using(function (): string {
        return "Orders can be returned within 30 days of delivery. " .
               "Refunds process within 3 to 5 business days. " .
               "Items must be unused and in original packaging.";
    });

$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withSystemPrompt(
        'You are a helpful customer support assistant for an online store. ' .
        'Use the available tools to look up order details, products, and policies. ' .
        'Always check actual data before making claims about orders or policies.'
    )
    ->withPrompt("I ordered something last week, order number ORD-2025-4821. " .
                 "Has it shipped yet? Also, what's your return policy?")
    ->withTools([$orderLookupTool, $productSearchTool, $refundPolicyTool])
    ->withMaxSteps(5)
    ->asText();

echo $response->text;

The withMaxSteps(5) call is important. It limits how many tool calls the model can make in a single request, preventing runaway chains where the model keeps calling tools indefinitely. Five steps is plenty for most support interactions.

What happens behind the scenes: the model reads the user message, decides it needs to call get_order_status with order number ORD-2025-4821, Prism runs your PHP function, returns the result to the model, the model sees the shipping status, then calls get_refund_policy for the second question, gets that result, and writes a complete response covering both questions using real data from your application.

Part 3: Embeddings for Semantic Search

Embeddings convert text into a vector, a list of floating point numbers that represent the semantic meaning of the text. Two pieces of text that mean similar things will have vectors that are close together in that high-dimensional space, even if the actual words used are completely different.

Prism handles embeddings through the same clean interface as text generation.

<?php

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

// Generate an embedding for a piece of text
$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-small')
    ->fromInput('How do I reset my account password?')
    ->create();

$vector = $response->embeddings[0]->embedding;
// $vector is an array of 1536 floats representing the semantic meaning

You can also embed multiple texts in a single API call, which is more efficient when indexing content:

<?php

$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-small')
    ->fromInput([
        'How do I reset my password?',
        'Account recovery steps for locked accounts',
        'Changing your email address in account settings',
        'Two-factor authentication setup guide',
    ])
    ->create();

foreach ($response->embeddings as $index => $embedding) {
    echo "Text {$index}: " . count($embedding->embedding) . " dimensions\n";
}

Real-World Example: Tying All Three Together

Here is where it gets practical. I will build a customer support assistant that uses all three Prism features together: embeddings to find relevant knowledge base articles, tool calling to look up live order data, and structured text generation to produce consistent responses.

The scenario is a support bot for a Laravel e-commerce application. The bot needs to answer questions about orders using real database data, find relevant help articles using semantic search, and produce responses that follow a consistent format.

Database Setup for Knowledge Base

php artisan make:migration create_knowledge_base_table

<?php

Schema::create('knowledge_base_articles', function (Blueprint $table) {
    $table->id();
    $table->string('title');
    $table->text('content');
    $table->json('embedding')->nullable();
    $table->string('category');
    $table->timestamps();
});

The Knowledge Base Indexer

<?php

namespace App\Services;

use App\Models\KnowledgeBaseArticle;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

class KnowledgeBaseIndexer
{
    public function indexAll(): void
    {
        $articles = KnowledgeBaseArticle::whereNull('embedding')->get();

        foreach ($articles->chunk(20) as $batch) {
            $texts = $batch->map(fn($a) => $a->title . "\n\n" . $a->content)
                           ->toArray();

            $response = Prism::embeddings()
                ->using(Provider::OpenAI, 'text-embedding-3-small')
                ->fromInput($texts)
                ->create();

            foreach ($batch as $index => $article) {
                $article->update([
                    'embedding' => $response->embeddings[$index]->embedding,
                ]);
            }

            usleep(200000);
        }
    }

    public function findRelevant(string $query, int $topK = 3): array
    {
        $queryResponse = Prism::embeddings()
            ->using(Provider::OpenAI, 'text-embedding-3-small')
            ->fromInput($query)
            ->create();

        $queryVector = $queryResponse->embeddings[0]->embedding;

        // Load articles and compute cosine similarity in PHP
        // For production with large knowledge bases, use pgvector instead
        $articles = KnowledgeBaseArticle::whereNotNull('embedding')->get();

        $scored = $articles->map(function ($article) use ($queryVector) {
            $articleVector = $article->embedding;
            return [
                'article'    => $article,
                'similarity' => $this->cosineSimilarity($queryVector, $articleVector),
            ];
        })
        ->filter(fn($item) => $item['similarity'] > 0.75)
        ->sortByDesc('similarity')
        ->take($topK);

        return $scored->pluck('article')->toArray();
    }

    private function cosineSimilarity(array $a, array $b): float
    {
        $dot  = array_sum(array_map(fn($x, $y) => $x * $y, $a, $b));
        $magA = sqrt(array_sum(array_map(fn($x) => $x ** 2, $a)));
        $magB = sqrt(array_sum(array_map(fn($x) => $x ** 2, $b)));

        return ($magA * $magB) > 0 ? $dot / ($magA * $magB) : 0.0;
    }
}

The Support Assistant Service

<?php

namespace App\Services;

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Tool;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\ArraySchema;

class CustomerSupportAssistant
{
    public function __construct(
        private KnowledgeBaseIndexer $knowledgeBase
    ) {}

    public function respond(string $customerMessage, string $customerId): array
    {
        // Step 1: Find relevant knowledge base articles using embeddings
        $relevantArticles = $this->knowledgeBase->findRelevant($customerMessage);

        $knowledgeContext = collect($relevantArticles)
            ->map(fn($a) => "### {$a->title}\n{$a->content}")
            ->join("\n\n---\n\n");

        // Step 2: Define tools for live data access
        $orderTool = Tool::as('get_order')
            ->for('Look up order details and status for a specific order number')
            ->withStringParameter('order_number', 'The order number, e.g. ORD-2025-1234')
            ->using(function (string $order_number) use ($customerId): string {
                $order = \App\Models\Order::where('order_number', $order_number)
                    ->where('customer_id', $customerId)
                    ->with('items', 'shipment')
                    ->first();

                if (!$order) {
                    return json_encode([
                        'error' => 'Order not found or does not belong to this customer',
                    ]);
                }

                return json_encode([
                    'order_number' => $order->order_number,
                    'status'       => $order->status,
                    'placed_at'    => $order->created_at->format('d M Y'),
                    'total'        => '$' . number_format($order->total, 2),
                    'item_count'   => $order->items->count(),
                    'tracking'     => $order->shipment->tracking_number ?? 'Not yet assigned',
                    'carrier'      => $order->shipment->carrier ?? null,
                    'estimated_delivery' => $order->shipment->estimated_delivery ?? null,
                ]);
            });

        $accountTool = Tool::as('get_account_info')
            ->for('Retrieve customer account information like email and membership status')
            ->using(function () use ($customerId): string {
                $customer = \App\Models\Customer::find($customerId);

                if (!$customer) {
                    return json_encode(['error' => 'Customer not found']);
                }

                return json_encode([
                    'name'              => $customer->name,
                    'email'             => $customer->email,
                    'member_since'      => $customer->created_at->format('M Y'),
                    'membership_tier'   => $customer->tier,
                    'total_orders'      => $customer->orders()->count(),
                ]);
            });

        // Step 3: Define schema for structured response
        $responseSchema = new ObjectSchema(
            name: 'support_response',
            description: 'A structured customer support response',
            properties: [
                new StringSchema('message', 'The response message to send to the customer'),
                new StringSchema(
                    'escalation_level',
                    'Whether to escalate: "none", "agent", or "manager"'
                ),
                new StringSchema('sentiment', 'Customer sentiment detected: "positive", "neutral", "frustrated"'),
                new ArraySchema(
                    'follow_up_actions',
                    'Actions the support team should take after this interaction',
                    new StringSchema('action', 'A follow-up action item')
                ),
            ],
            requiredFields: ['message', 'escalation_level', 'sentiment']
        );

        // Step 4: Generate the response using text generation, tools, and schema
        $systemPrompt = "You are a helpful and empathetic customer support assistant.

Relevant knowledge base articles for this conversation:
{$knowledgeContext}

Guidelines:
- Use the knowledge base content above when it is relevant to the customer's question.
- Use the available tools to look up live order and account data when needed.
- Keep responses concise and clear, two to three sentences where possible.
- If the customer is frustrated, acknowledge their feelings first before solving the problem.
- Escalate to 'agent' if the issue requires human judgment or account changes.
- Escalate to 'manager' only if the customer is threatening to leave or requests a manager specifically.
- Never guess at order details, always use the get_order tool for specific order questions.";

        $response = Prism::text()
            ->using(Provider::Anthropic, 'claude-3-7-sonnet-latest')
            ->withSystemPrompt($systemPrompt)
            ->withPrompt($customerMessage)
            ->withTools([$orderTool, $accountTool])
            ->withMaxSteps(4)
            ->withSchema($responseSchema)
            ->asStructured();

        return $response->structured;
    }
}

The Controller

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use App\Services\CustomerSupportAssistant;

class SupportController extends Controller
{
    public function __construct(
        private CustomerSupportAssistant $assistant
    ) {}

    public function chat(Request $request)
    {
        $request->validate([
            'message' => 'required|string|max:1000',
        ]);

        $customerId = auth()->id();
        $message    = $request->input('message');

        try {
            $response = $this->assistant->respond($message, $customerId);

            return response()->json([
                'reply'            => $response['message'],
                'escalation_level' => $response['escalation_level'],
                'sentiment'        => $response['sentiment'],
                'follow_up'        => $response['follow_up_actions'] ?? [],
            ]);

        } catch (\Exception $e) {
            report($e);
            return response()->json([
                'reply' => 'Something went wrong. Please try again in a moment.',
            ], 500);
        }
    }
}

What This System Produces

Here is a realistic example of what the full pipeline returns for a frustrated customer asking about a delayed order:

Customer message: "My order ORD-2025-4821 was supposed to arrive three days ago
and I still haven't received it. This is really frustrating."

System flow:
1. Embeddings search finds "Shipping delays FAQ" and "How to track your order" articles
2. Claude reads the relevant knowledge base content
3. Claude calls get_order tool with order number ORD-2025-4821
4. Tool returns: { status: "shipped", tracking: "TRK987654", carrier: "FedEx",
                   estimated_delivery: "2 days ago" }
5. Claude generates structured response

Response:
{
  "message": "I completely understand your frustration, and I am sorry your order
              is running late. I can see ORD-2025-4821 shipped with FedEx and the
              tracking number is TRK987654. FedEx is showing a delay on their end,
              but the package is still in transit. You can track it directly at
              fedex.com using that number for the most current status.",
  "escalation_level": "none",
  "sentiment": "frustrated",
  "follow_up_actions": [
    "Monitor order TRK987654 for delivery confirmation",
    "If not delivered within 48 hours, initiate trace request with FedEx",
    "Flag customer account for priority handling on next contact"
  ]
}

The sentiment flag lets your frontend show a different UI for frustrated customers. The escalation level drives routing logic. The follow-up actions can be stored and assigned to your support team automatically. This is not just a chatbot, it is a complete support workflow powered by three Prism features working together.

Testing Prism Code

One of the things that makes Prism genuinely production-ready is its testing utilities. You do not want real API calls firing during unit tests. Prism ships with response faking so you can test your application logic without hitting any external APIs.

<?php

use Prism\Prism\Facades\Prism;
use Prism\Prism\Testing\PrismFake;
use Prism\Prism\ValueObjects\TextResult;

it('generates a support response for order queries', function () {
    $fakeResponse = new TextResult(
        text: '{"message": "Your order has shipped.", "escalation_level": "none", "sentiment": "neutral"}',
        finishReason: 'stop',
        usage: ['prompt_tokens' => 100, 'completion_tokens' => 50]
    );

    Prism::fake([$fakeResponse]);

    $assistant = app(CustomerSupportAssistant::class);
    $result    = $assistant->respond('Where is my order?', customerId: 1);

    expect($result['escalation_level'])->toBe('none');
    expect($result['sentiment'])->toBe('neutral');

    Prism::assertCallCount(1);
    Prism::assertLastCallUsedProvider('anthropic');
});

Prism::fake() intercepts all Prism calls and returns your predefined responses. Prism::assertCallCount() and Prism::assertLastCallUsedProvider() let you verify your code is making the right calls. Clean, straightforward, and no real API usage during tests.

Switching Providers Without Touching Application Code

One last thing worth showing explicitly, because it is the whole point of Prism. You can make your provider configurable through your .env file so you can switch without a code change:

# .env
AI_PROVIDER=anthropic
AI_MODEL=claude-3-7-sonnet-latest

<?php

// In your service or controller
$response = Prism::text()
    ->using(
        config('ai.provider', 'openai'),
        config('ai.model', 'gpt-4o')
    )
    ->withPrompt($prompt)
    ->asText();

Change the provider in .env, restart the queue workers, done. No code changes, no redeployment of application logic. That is the practical benefit of a unified interface. You build once against Prism's API and gain the flexibility to move between providers as your needs evolve, pricing changes, or a new model comes along that performs better on your specific tasks.

Prism is still relatively young but it is actively maintained and the API has stabilised enough to build production features on. For any new Laravel project that involves AI, it is the first package I reach for now. The alternative, direct API clients for each provider, creates the kind of fragmented codebase that becomes a maintenance problem fast.

Joomla's built-in search has always had the same fundamental limitation. It is keyword-based. A visitor types "how do I reset my account" and the search engine looks for articles containing those exact words. If your article uses the phrase "recover your login credentials" instead, it does not show up. The visitor gets no results, concludes your site does not have the answer, and leaves.

This is not a Joomla problem specifically. It is what keyword search does. It matches strings, not meaning. RAG, Retrieval-Augmented Generation, solves this at the architecture level. Instead of matching keywords, it converts both your content and the search query into vector embeddings, finds content that is semantically similar, and uses an LLM to generate a direct answer from that content. A visitor asking "how do I reset my account" gets a proper answer even if none of your articles use those exact words.

I will walk through the full implementation. We will cover the three main vector storage options honestly so you can make the right choice for your setup, then go deep on building the complete RAG pipeline inside a custom Joomla component using PostgreSQL with pgvector and OpenAI.

What you need: Joomla 4 or 5, PHP 8.1+, Composer, PostgreSQL with the pgvector extension installed, and an OpenAI API key.

What RAG Actually Does, Step by Step

Before writing any code it is worth being clear about what the pipeline actually does at each stage. RAG is one of those terms that gets thrown around loosely and the implementation details matter.

Phase 1: Indexing (runs once, then on content updates)
        ↓
Fetch all published Joomla articles
        ↓
Split long articles into chunks
        ↓
Send each chunk to OpenAI Embeddings API
        ↓
Store the chunk text and its embedding vector in PostgreSQL

Phase 2: Search (runs on every user query)
        ↓
User submits a search query
        ↓
Convert query to an embedding vector via OpenAI
        ↓
Find the most semantically similar chunks using pgvector
        ↓
Send retrieved chunks plus the original query to GPT-4o
        ↓
GPT-4o generates a direct answer grounded in your content
        ↓
Return the answer and source article links to the user

The indexing phase is the slower one and only needs to run when content changes. The search phase is what your visitors experience and it needs to be fast. Keeping those two concerns separate in the architecture makes both easier to manage.

Three Vector Storage Options for Joomla

This is the decision that shapes the rest of the implementation. There is no universally correct answer here, the right choice depends on your infrastructure, team, and content volume.

Option 1: PostgreSQL with pgvector

pgvector is an open source PostgreSQL extension that adds a native vector data type and similarity search operators. You store embeddings directly in a PostgreSQL table alongside your chunk text and metadata. Similarity search runs as a standard SQL query using the cosine distance operator.

The big advantage is that you are not adding a new infrastructure dependency. If you are already running PostgreSQL, this is just an extension install and a new table. Queries are fast, the data lives in your existing database stack, and you have full control. The limitation is that at very large scale, hundreds of thousands of chunks, you need to tune the index carefully to maintain query speed.

This is what we are building in this post. It is the right default for most Joomla sites.

Option 2: MySQL with a Vector Similarity Workaround

Joomla ships with MySQL as its default database, so this is the path of least resistance from an infrastructure standpoint. MySQL 9.0 added experimental vector support but it is not production-ready for most use cases yet. The practical workaround is to store embeddings as JSON or a serialised float array, fetch candidate chunks using a broad text filter, then do the cosine similarity calculation in PHP.

This works for small content sets, a few hundred articles. It gets slow quickly as the content volume grows because you are doing similarity math in PHP rather than in an optimised database index. If your Joomla site runs MySQL and you cannot add PostgreSQL, this is a viable starting point but plan for a migration if the search volume grows.

Option 3: External Vector Store, Pinecone or Qdrant

Pinecone and Qdrant are purpose-built vector databases. You send embeddings to their API, they handle storage and indexing, and you query them via HTTP. Both have generous free tiers for getting started.

The advantage is performance at scale and zero infrastructure management on your end. The disadvantages are an additional external dependency, data leaving your infrastructure, API rate limits, and ongoing costs that grow with your content volume. For enterprise Joomla sites with strict data residency requirements, an external service is often a non-starter.

Good fit for teams that want to move fast without managing PostgreSQL, or sites with very high search volume where a dedicated vector store makes sense operationally.

We are going with pgvector. Here is the full build.

Install pgvector and Set Up the Database Table

First, install the pgvector extension in your PostgreSQL database. If you have superuser access:

CREATE EXTENSION IF NOT EXISTS vector;

If you are on a managed PostgreSQL service like AWS RDS or Supabase, pgvector is available as an enabled extension in the console without needing superuser access.

Create the table that will store your article chunks and their embeddings. Run this in your PostgreSQL database, this is a separate database from Joomla's MySQL database:

CREATE TABLE joomla_article_embeddings (
    id           SERIAL PRIMARY KEY,
    article_id   INTEGER NOT NULL,
    article_title TEXT NOT NULL,
    chunk_index  INTEGER NOT NULL,
    chunk_text   TEXT NOT NULL,
    embedding    vector(1536),
    url          TEXT,
    created_at   TIMESTAMP DEFAULT NOW(),
    updated_at   TIMESTAMP DEFAULT NOW()
);

CREATE INDEX ON joomla_article_embeddings
    USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);

CREATE INDEX ON joomla_article_embeddings (article_id);

The embedding dimension is 1536 because that is what OpenAI's text-embedding-3-small model outputs. If you use text-embedding-3-large instead, change this to 3072. The ivfflat index is what makes similarity search fast at scale. The lists value of 100 is a reasonable starting point, tune it upward if you have more than 100,000 chunks.

Custom Joomla Component Structure

We will build this as a custom Joomla component. Create the following structure under components/com_ragsearch:

components/com_ragsearch/
    ragsearch.xml
    src/
        Service/
            OpenAIService.php
            VectorStoreService.php
            ArticleChunkerService.php
            RAGSearchService.php
        Controller/
            SearchController.php
        View/
            Search/
                HtmlView.php
                tmpl/
                    default.php
    tmpl/
        index.php

administrator/components/com_ragsearch/
    src/
        Controller/
            IndexController.php

Install the OpenAI PHP client and a PostgreSQL driver via Composer in your Joomla root:

composer require openai-php/client
composer require doctrine/dbal

The OpenAI Service

Create src/Service/OpenAIService.php:

<?php

namespace Joomla\Component\Ragsearch\Site\Service;

use OpenAI;

class OpenAIService
{
    private $client;

    public function __construct()
    {
        $params      = \JComponentHelper::getParams('com_ragsearch');
        $apiKey      = $params->get('openai_api_key');
        $this->client = OpenAI::client($apiKey);
    }

    public function embed(string $text): array
    {
        $response = $this->client->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $text,
        ]);

        return $response->embeddings[0]->embedding;
    }

    public function embedBatch(array $texts): array
    {
        $response = $this->client->embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $texts,
        ]);

        $embeddings = [];
        foreach ($response->embeddings as $item) {
            $embeddings[$item->index] = $item->embedding;
        }

        return $embeddings;
    }

    public function generateAnswer(string $query, array $chunks): string
    {
        $context = implode("\n\n---\n\n", array_column($chunks, 'chunk_text'));

        $response = $this->client->chat()->create([
            'model'       => 'gpt-4o',
            'temperature' => 0.3,
            'max_tokens'  => 600,
            'messages'    => [
                [
                    'role'    => 'system',
                    'content' => 'You are a helpful site assistant. Answer the user question
                                  using only the content provided below. If the content does
                                  not contain enough information to answer, say so honestly.
                                  Do not make up information. Keep answers clear and concise.',
                ],
                [
                    'role'    => 'user',
                    'content' => "Content from our site:\n\n{$context}\n\nQuestion: {$query}",
                ],
            ],
        ]);

        return $response->choices[0]->message->content;
    }
}

Notice the embedBatch method. When indexing articles, sending texts in batches rather than one at a time cuts the number of API calls significantly and speeds up the indexing process. Use it during the indexing phase, use embed for single query embeddings at search time.

The Article Chunker Service

Long articles need to be split into chunks before embedding. Embedding an entire 3,000-word article as a single vector produces a representation that is too diffuse to be useful for retrieval. Smaller focused chunks give the similarity search something meaningful to match against.

Create src/Service/ArticleChunkerService.php:

<?php

namespace Joomla\Component\Ragsearch\Site\Service;

class ArticleChunkerService
{
    private int $chunkSize    = 400;
    private int $chunkOverlap = 50;

    public function chunk(string $text): array
    {
        // Strip HTML tags from article body
        $clean = strip_tags($text);

        // Normalise whitespace
        $clean = preg_replace('/\s+/', ' ', $clean);
        $clean = trim($clean);

        $words  = explode(' ', $clean);
        $total  = count($words);
        $chunks = [];
        $start  = 0;

        while ($start < $total) {
            $end        = min($start + $this->chunkSize, $total);
            $chunkWords = array_slice($words, $start, $end - $start);
            $chunks[]   = implode(' ', $chunkWords);

            // Move forward by chunkSize minus overlap
            // so consecutive chunks share some context
            $start += ($this->chunkSize - $this->chunkOverlap);

            if ($start >= $total) {
                break;
            }
        }

        return array_filter($chunks, fn($c) => strlen(trim($c)) > 50);
    }
}

The overlap between chunks matters more than it might seem. If a key sentence sits right at the boundary between two chunks, without overlap it gets split in half and neither chunk represents that idea well. A 50-word overlap means boundary content appears in both adjacent chunks, so the similarity search is more likely to retrieve it when it is relevant.

The Vector Store Service

Create src/Service/VectorStoreService.php:

<?php

namespace Joomla\Component\Ragsearch\Site\Service;

use Doctrine\DBAL\DriverManager;

class VectorStoreService
{
    private $conn;

    public function __construct()
    {
        $params      = \JComponentHelper::getParams('com_ragsearch');

        $this->conn = DriverManager::getConnection([
            'dbname'   => $params->get('pg_database'),
            'user'     => $params->get('pg_user'),
            'password' => $params->get('pg_password'),
            'host'     => $params->get('pg_host', 'localhost'),
            'port'     => $params->get('pg_port', 5432),
            'driver'   => 'pdo_pgsql',
        ]);
    }

    public function upsertChunk(
        int    $articleId,
        string $title,
        int    $chunkIndex,
        string $chunkText,
        array  $embedding,
        string $url
    ): void {
        // Delete existing chunks for this article and index first
        $this->conn->executeStatement(
            'DELETE FROM joomla_article_embeddings
             WHERE article_id = :id AND chunk_index = :idx',
            ['id' => $articleId, 'idx' => $chunkIndex]
        );

        $vectorLiteral = '[' . implode(',', $embedding) . ']';

        $this->conn->executeStatement(
            'INSERT INTO joomla_article_embeddings
                (article_id, article_title, chunk_index, chunk_text, embedding, url)
             VALUES
                (:article_id, :title, :chunk_index, :chunk_text, :embedding, :url)',
            [
                'article_id'  => $articleId,
                'title'       => $title,
                'chunk_index' => $chunkIndex,
                'chunk_text'  => $chunkText,
                'embedding'   => $vectorLiteral,
                'url'         => $url,
            ]
        );
    }

    public function similaritySearch(array $queryEmbedding, int $topK = 5): array
    {
        $vectorLiteral = '[' . implode(',', $queryEmbedding) . ']';

        $sql = "SELECT
                    article_id,
                    article_title,
                    chunk_text,
                    url,
                    1 - (embedding <=> :embedding::vector) AS similarity
                FROM joomla_article_embeddings
                ORDER BY embedding <=> :embedding::vector
                LIMIT :limit";

        $stmt = $this->conn->executeQuery(
            $sql,
            [
                'embedding' => $vectorLiteral,
                'limit'     => $topK,
            ]
        );

        return $stmt->fetchAllAssociative();
    }

    public function deleteArticle(int $articleId): void
    {
        $this->conn->executeStatement(
            'DELETE FROM joomla_article_embeddings WHERE article_id = :id',
            ['id' => $articleId]
        );
    }
}

The <=> operator is pgvector's cosine distance operator. Cosine distance measures the angle between two vectors rather than the straight-line distance between them, which works better for text embeddings because it focuses on direction, meaning, rather than magnitude. The similarity score in the SELECT is calculated as 1 - cosine_distance, so a score of 1.0 is a perfect match and 0.0 is completely unrelated.

The Indexing Controller

This runs from the Joomla administrator backend. It fetches all published articles, chunks them, embeds them in batches, and stores everything in PostgreSQL. You run this once to build the initial index and then on a schedule or via a hook when articles are updated.

Create administrator/components/com_ragsearch/src/Controller/IndexController.php:

<?php

namespace Joomla\Component\Ragsearch\Administrator\Controller;

use Joomla\CMS\MVC\Controller\BaseController;
use Joomla\Component\Ragsearch\Site\Service\OpenAIService;
use Joomla\Component\Ragsearch\Site\Service\VectorStoreService;
use Joomla\Component\Ragsearch\Site\Service\ArticleChunkerService;

class IndexController extends BaseController
{
    private int $batchSize = 20;

    public function build(): void
    {
        $db      = $this->app->getDatabase();
        $chunker = new ArticleChunkerService();
        $openai  = new OpenAIService();
        $store   = new VectorStoreService();

        // Fetch all published Joomla articles
        $query = $db->getQuery(true)
            ->select(['a.id', 'a.title', 'a.introtext', 'a.fulltext'])
            ->from($db->quoteName('#__content', 'a'))
            ->where($db->quoteName('a.state') . ' = 1');

        $articles = $db->setQuery($query)->loadObjectList();

        $indexed = 0;
        $errors  = 0;

        foreach ($articles as $article) {
            try {
                $fullContent = $article->title . "\n\n"
                             . strip_tags($article->introtext) . "\n\n"
                             . strip_tags($article->fulltext);

                $chunks = $chunker->chunk($fullContent);

                if (empty($chunks)) {
                    continue;
                }

                $url = \JRoute::_(
                    'index.php?option=com_content&view=article&id=' . $article->id
                );

                // Delete old embeddings for this article before re-indexing
                $store->deleteArticle($article->id);

                // Process chunks in batches to reduce API calls
                $chunkBatches = array_chunk($chunks, $this->batchSize);

                foreach ($chunkBatches as $batchIndex => $batch) {
                    $embeddings = $openai->embedBatch($batch);

                    foreach ($batch as $i => $chunkText) {
                        $globalIndex = ($batchIndex * $this->batchSize) + $i;
                        $embedding   = $embeddings[$i] ?? null;

                        if (!$embedding) {
                            continue;
                        }

                        $store->upsertChunk(
                            $article->id,
                            $article->title,
                            $globalIndex,
                            $chunkText,
                            $embedding,
                            $url
                        );
                    }

                    // Small pause between batches to stay within API rate limits
                    usleep(200000);
                }

                $indexed++;

            } catch (\Exception $e) {
                $errors++;
                \JLog::add(
                    'RAG indexing failed for article ' . $article->id . ': ' . $e->getMessage(),
                    \JLog::ERROR,
                    'com_ragsearch'
                );
            }
        }

        $this->app->enqueueMessage(
            "Indexing complete. Articles indexed: {$indexed}. Errors: {$errors}.",
            $errors > 0 ? 'warning' : 'success'
        );

        $this->setRedirect('index.php?option=com_ragsearch');
    }
}

The RAG Search Service

Create src/Service/RAGSearchService.php:

<?php

namespace Joomla\Component\Ragsearch\Site\Service;

class RAGSearchService
{
    public function __construct(
        private OpenAIService     $openai,
        private VectorStoreService $store
    ) {}

    public function search(string $query): array
    {
        if (strlen(trim($query)) < 3) {
            return [
                'answer'  => 'Please enter a more specific question.',
                'sources' => [],
            ];
        }

        // Convert the query to a vector embedding
        $queryEmbedding = $this->openai->embed($query);

        // Find the most semantically similar chunks
        $chunks = $this->store->similaritySearch($queryEmbedding, topK: 5);

        if (empty($chunks)) {
            return [
                'answer'  => 'No relevant content found for your query. Try rephrasing your question.',
                'sources' => [],
            ];
        }

        // Filter out low-similarity results
        $relevantChunks = array_filter(
            $chunks,
            fn($c) => ($c['similarity'] ?? 0) > 0.75
        );

        if (empty($relevantChunks)) {
            return [
                'answer'  => 'I could not find content closely matching your question. Please try different keywords.',
                'sources' => [],
            ];
        }

        // Generate a direct answer grounded in the retrieved chunks
        $answer = $this->openai->generateAnswer($query, $relevantChunks);

        // Deduplicate sources by article ID
        $sources = [];
        foreach ($relevantChunks as $chunk) {
            $aid = $chunk['article_id'];
            if (!isset($sources[$aid])) {
                $sources[$aid] = [
                    'title' => $chunk['article_title'],
                    'url'   => $chunk['url'],
                ];
            }
        }

        return [
            'answer'  => $answer,
            'sources' => array_values($sources),
        ];
    }
}

The similarity threshold of 0.75 is worth paying attention to. Below that score the retrieved chunks are probably not relevant enough to be useful for generating an answer. You can adjust this up or down depending on how your content is structured and how specific the queries on your site tend to be. Start at 0.75 and tune based on real search results.

The Search Controller and View

Create src/Controller/SearchController.php:

<?php

namespace Joomla\Component\Ragsearch\Site\Controller;

use Joomla\CMS\MVC\Controller\BaseController;
use Joomla\Component\Ragsearch\Site\Service\OpenAIService;
use Joomla\Component\Ragsearch\Site\Service\VectorStoreService;
use Joomla\Component\Ragsearch\Site\Service\RAGSearchService;

class SearchController extends BaseController
{
    public function search(): void
    {
        $query = trim($this->input->getString('q', ''));

        $result = ['answer' => '', 'sources' => [], 'query' => $query];

        if (!empty($query)) {
            try {
                $service = new RAGSearchService(
                    new OpenAIService(),
                    new VectorStoreService()
                );

                $result = array_merge($result, $service->search($query));

            } catch (\Exception $e) {
                $result['answer'] = 'Search is temporarily unavailable. Please try again shortly.';
                \JLog::add('RAG search error: ' . $e->getMessage(), \JLog::ERROR, 'com_ragsearch');
            }
        }

        $this->app->setUserState('com_ragsearch.result', $result);
        $this->setRedirect(\JRoute::_('index.php?option=com_ragsearch&view=search'));
    }
}

The Blade-equivalent Joomla view template at src/View/Search/tmpl/default.php:

<?php defined('_JEXEC') or die; ?>

<div class="rag-search">

    <form method="POST" action="<?php echo JRoute::_('index.php?option=com_ragsearch&task=search.search'); ?>">
        <?php echo JHtml::_('form.token'); ?>
        <input type="text"
               name="q"
               value="<?php echo htmlspecialchars($this->result['query'] ?? ''); ?>"
               placeholder="Ask anything about our site..."
               autocomplete="off">
        <button type="submit">Search</button>
    </form>

    <?php if (!empty($this->result['answer'])) : ?>

        <div class="rag-answer">
            <h3>Answer</h3>
            <p><?php echo nl2br(htmlspecialchars($this->result['answer'])); ?></p>
        </div>

        <?php if (!empty($this->result['sources'])) : ?>
            <div class="rag-sources">
                <h4>Sources</h4>
                <ul>
                    <?php foreach ($this->result['sources'] as $source) : ?>
                        <li>
                            <a href="<?php echo htmlspecialchars($source['url']); ?>">
                                <?php echo htmlspecialchars($source['title']); ?>
                            </a>
                        </li>
                    <?php endforeach; ?>
                </ul>
            </div>
        <?php endif; ?>

    <?php endif; ?>

</div>

Keeping the Index fresh

The index goes stale the moment an article is updated and not re-indexed. There are two clean ways to handle this in Joomla.

The first is a Joomla plugin that hooks into onContentAfterSave and triggers re-indexing for the saved article specifically. This keeps the index fresh in real time but adds latency to every article save operation.

<?php

class PlgContentRagsearchIndex extends JPlugin
{
    public function onContentAfterSave(
        string $context,
        object $article,
        bool   $isNew
    ): void {
        if ($context !== 'com_content.article') {
            return;
        }

        if ((int) $article->state !== 1) {
            return;
        }

        // Dispatch a Joomla queue task instead of indexing synchronously
        // to avoid blocking the article save response
        \Joomla\CMS\Queue\QueueFacade::push('ragsearch.index_article', [
            'article_id' => $article->id,
        ]);
    }
}

The second approach is a scheduled CLI task that re-indexes all articles on a schedule, say every hour or every night. For sites where content does not change frequently, a nightly re-index via Joomla's task scheduler is simpler and puts zero overhead on the save operation.

For most sites the scheduled approach is the right default. Use the plugin approach only if your content changes continuously throughout the day and freshness matters within minutes.

What this looks like for a Real Visitor

Here is a concrete example. Say your Joomla site has articles about software products and a visitor types: "what happens if I cancel my subscription mid-month?"

Your articles probably use phrases like "pro-rata refund policy", "billing cycle", "account downgrade", not the exact words the visitor used. Keyword search returns nothing. The RAG pipeline converts the query to a vector, finds three chunks from your billing and account articles that are semantically close to that question, feeds them to GPT-4o, and returns something like:

"If you cancel mid-month, your account remains active until the end of your current billing period. You will not be charged for the following month. Refunds for unused days are not issued automatically but can be requested within 7 days of cancellation by contacting support."

Below the answer, the visitor sees links to the two source articles that information came from. They got a direct answer, they can read the full policy if they want to, and they did not have to trawl through search results guessing which article might be relevant.

A Few things to know before Go Live

The embedding cost for the initial indexing run is usually smaller than people expect. A site with 500 articles at 400 words each, split into chunks of 400 words with 50-word overlap, produces roughly 600 to 700 chunks. At OpenAI's current pricing for text-embedding-3-small, that initial index costs well under a dollar. Ongoing costs per search query are minimal, one embedding call per query.

Caching search results is worth adding early. Many visitors on the same site ask very similar questions. Store recent query-answer pairs in Joomla's cache layer with a TTL of a few hours. The cache hit rate on popular queries tends to be high and it cuts both API costs and response time meaningfully.

Finally, keep an eye on what people actually search for. Log the queries, log whether the similarity search returned results above the threshold, and log whether users clicked the source links. After a few weeks you will see which questions the pipeline handles well and which ones consistently miss. That data tells you whether your chunking strategy needs adjusting, whether your similarity threshold is set correctly, and whether there are content gaps on your site worth addressing.

The RAG pattern is one of the most practically useful things you can add to a content-heavy Joomla site. It turns a search box that frustrates visitors into one that actually helps them find what they need, in their own words, without requiring your content to match their exact phrasing.