# Building Your First AI Service with Quarkus, LangChain4j, and Ollama

> February 24, 2026 · https://eldermoraes.com/building-your-first-ai-service-with-quarkus-langchain4j-and-ollama/

* * *

The real value of AI in the enterprise lies in what an application can do. Integrating Large Language Models (LLMs) into Java applications is no longer a complex task. With Quarkus and LangChain4j, you can build a production-ready AI service that runs entirely on your local hardware using Ollama.

## 1\. Introduction

For Java developers, the focus has shifted from what an LLM knows to how we can use that knowledge within our systems. This tutorial guides you through creating a local AI service that uses Retrieval-Augmented Generation (RAG) to answer questions based on a custom dataset.

## 2\. Prerequisites

- **JDK 21+**: Required for the latest Quarkus features.
    
- **Maven 3.9+**: For project builds and dependency management.
    
- **Ollama**: Download and install from [ollama.com](https://ollama.com/).
    
- **Cloud Model**: Run `ollama run gpt-oss:120b-cloud` so you can establish the connection with this model on Ollama Cloud.
    

## 3\. Project Setup

Use the Quarkus Maven plugin to generate a project named `first-chatbot` with the necessary extensions :

```
mvn "io.quarkus.platform:quarkus-maven-plugin:create" \
    -DprojectGroupId=com.eldermoraes \
    -DprojectArtifactId=first-chatbot \
    -Dextensions="langchain4j-ollama,langchain4j-easy-rag,quarkus-rest,quarkus-smallrye-openapi"
```

### Key Extensions

- **quarkus-rest**: Provides the modern Jakarta REST implementation.
    
- **langchain4j-ollama**: Connects your application to the Ollama API.
    
- **langchain4j-easy-rag**: Handles document ingestion and retrieval automatically.
    
- **quarkus-smallrye-openapi**: Generates Swagger UI for easy testing.
    

## 4\. Exploring Quarkus, LangChain4j, and Ollama

These three components form a powerful stack :

- **Quarkus**: The engine that handles dependency injection and lifecycle management.
    
- **LangChain4j**: The orchestrator that allows you to define AI behavior via Java interfaces.
    
- **Ollama**: The brain that runs the LLM locally on your machine.
    

## 5\. Implementing Your First AI Service

### Step 1: Prepare Your Knowledge Base

Create a directory at `src/main/resources/rag`. You need data files for the RAG process to work. For this tutorial, download the Star Wars dataset (JSON files) from this repository: [swapi-app data](https://github.com/eldermoraes/swapi.build/tree/main/swapi-app/src/main/resources/data).

Place the downloaded files into your `src/main/resources/rag` folder.

### Step 2: Configure the Application

Edit `src/main/resources/application.properties` to wire the components together.

```
# Ollama Model Configuration
quarkus.langchain4j.ollama.chat-model.model-id=gpt-oss:120b-cloud
quarkus.langchain4j.timeout=60s

# Easy RAG Path
quarkus.langchain4j.easy-rag.path=src/main/resources/rag
```

### Step 3: Create the AI Service

Define an interface for your assistant. The `@RegisterAiService` annotation tells Quarkus to implement the logic for you.

```
package com.eldermoraes;

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;
import jakarta.enterprise.context.ApplicationScoped;

@RegisterAiService
@ApplicationScoped
public interface StarWarsAssistant {

    @SystemMessage("""
        You are an expert on the Star Wars universe. 
        Use the provided information to answer questions concisely.
        """)
    String chat(@UserMessage String question);
}
```

### Step 4: Expose the REST Endpoint

Inject the service into a REST resource to make it accessible :

```
package com.eldermoraes;

import jakarta.inject.Inject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.QueryParam;

@Path("/star-wars")
public class StarWarsResource {

    @Inject
    StarWarsAssistant assistant;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String ask(@QueryParam("question") String question) {
        return assistant.chat(question);
    }
}
```

## 6\. Testing and Running the Service

Start your application in development mode :

```
./mvnw quarkus:dev
```

Quarkus will automatically ingest the files from your `rag` folder. Open your browser to `http://localhost:8080/q/swagger-ui`. Locate the `/star-wars` endpoint and try a query like: _"What is the climate on the planet Tatooine?"_

The service will retrieve the relevant information from your local JSON files and provide an answer grounded in that data.

## 7\. Conclusion

You now have a functional AI service running locally. This setup provides zero API costs.

To expand your service, consider:

- **Switching Models**: Try other models to compare different behaviours.
    
- **Adding your own stuff**: Try building a service around some topic you like (Lord of the Rings? Stranger Things? Magic: The Gathering?). It's much more fun!