Embeddings without the hype: pgvector and .NET

Every system with free-text input grows the same weed. Users create entries that already exist, just worded differently:

• “Kickoff meeting notes”
• “Meeting 1 results”
• “Project start discussion”

Three labels, one meeting. String comparison sees three distinct values; a unique constraint is useless; and full-text search only helps if the words overlap, which here they barely do. I’ve hit this in several projects, and the fix that stuck is almost embarrassingly small: turn the text into an embedding, compare it against what’s already stored, and refuse to save anything that means the same thing. No model training, no prompt engineering, one short C# program.

Embeddings in one paragraph

An embedding model maps a piece of text to a vector, a few hundred or thousand floating-point numbers, such that texts with similar meaning land close together. “Kickoff meeting notes” and “Project start discussion” share almost no words, but their vectors sit nearly on top of each other. Closeness is measured with cosine distance: 0 means “same direction, same meaning”, higher means less related. That’s the entire theory this post needs. Two practical notes: the models are multilingual (a German label and its English twin land close together too, which matters in mixed-language enterprise data), and generating an embedding is a cheap, fast API call, not a chat completion.

The whole program

Three packages: Microsoft.Extensions.AI (the abstraction), Azure.AI.OpenAI with Microsoft.Extensions.AI.OpenAI (the provider), and Pgvector.EntityFrameworkCore (vectors as a first-class EF Core type on PostgreSQL). First the wiring:

var builder = Host.CreateApplicationBuilder(args);

// 1. the embedding client, behind the M.E.AI abstraction
builder.Services.AddEmbeddingGenerator(_ =>
    new AzureOpenAIClient(
            new Uri("https://your-endpoint.openai.azure.com/"),
            new ApiKeyCredential(builder.Configuration["OpenAI:ApiKey"]!))
        .GetEmbeddingClient("text-embedding-3-small")
        .AsIEmbeddingGenerator());

// 2. EF Core with the pgvector extension enabled
builder.Services.AddDbContext<AppDbContext>(options =>
    options.UseNpgsql(builder.Configuration.GetConnectionString("Db"),
        o => o.UseVector()));

The data model is one entity with a Vector column. text-embedding-3-small produces 1536 dimensions, and the column type says so:

class Document {
    public int Id { get; set; }
    public string Content { get; set; } = string.Empty;
    public Vector Embedding { get; set; } = null!;
}

class AppDbContext(DbContextOptions<AppDbContext> options) : DbContext(options) {
    public DbSet<Document> Documents => Set<Document>();

    protected override void OnModelCreating(ModelBuilder modelBuilder) {
        modelBuilder.HasPostgresExtension("vector");
        modelBuilder.Entity<Document>()
            .Property(d => d.Embedding)
            .HasColumnType("vector(1536)");
    }
}

And the core flow: embed the input, look for close neighbors, save only if nothing similar exists.

var generator = scope.ServiceProvider
    .GetRequiredService<IEmbeddingGenerator<string, Embedding<float>>>();
var embedding = await generator.GenerateAsync(text);
var vector = new Vector(embedding.Vector.ToArray());

var similar = await db.Documents
    .Where(d => d.Embedding.CosineDistance(vector) < 0.2)
    .OrderBy(d => d.Embedding.CosineDistance(vector))
    .ToListAsync();

if (similar.Count != 0) {
    Console.WriteLine("Similar entries already exist:");
    foreach (var d in similar) Console.WriteLine($"  {d.Content}");
} else {
    db.Documents.Add(new Document { Content = text, Embedding = vector });
    await db.SaveChangesAsync();
    Console.WriteLine($"Saved '{text}'");
}

CosineDistance translates straight to pgvector’s distance operator, so the comparison runs inside PostgreSQL, not in your application. Run it twice and the behavior is exactly what you want:

dotnet run "Kickoff meeting notes"
# Saved 'Kickoff meeting notes'

dotnet run "Project start discussion"
# Similar entries already exist:
#   Kickoff meeting notes

The database view after a few inserts is pleasantly boring: rows of text, each with its vector sitting in a regular column. No separate vector database, no sync pipeline. If you read my Just use Postgres post: pgvector is precisely the “extensions are composition, not scale-out” argument in action.

Tuning the gate

The one number that matters is the threshold. 0.2 cosine distance (roughly “80% similar”) is where I start for short labels, but treat it as a dial, not a truth:

• Too strict (say 0.05) and only near-identical strings match; the weed grows back.
• Too loose (say 0.4) and “Meeting notes April” swallows “Meeting notes May”, which users experience as data loss.
• Log before you block. Run the gate in observe-only mode for a week, record what would have been rejected, and read the log. The right threshold for your domain falls out of that list; mine have landed between 0.15 and 0.25.
• Cheap checks first. An exact-match or normalized-string check costs nothing and catches the trivial duplicates before you spend an API call.

And don’t hard-block on a soft signal: the honest UX is “this looks like it already exists, here’s what we found, save anyway?”, not a silent refusal.

The same trick powers search

Deduplication is embedding-distance with a threshold. Drop the threshold and take the top results instead, and you’ve built semantic search, which is where this quietly shines: dropdowns with hundreds of entries. Users don’t scroll through 400 articles; they type what they mean, and the nearest neighbors surface:

var results = await db.Documents
    .OrderBy(d => d.Embedding.CosineDistance(queryVector))
    .Take(5)
    .ToListAsync();

A user types “battery for sensor B” and the list offers “Item 4579-B (lithium, 3.7 V)”, because the embedding space knows those belong together even though no substring matches. That’s the feature people call “AI-powered UX” on slides; it’s an ORDER BY in code. Two production notes: debounce the input (one embedding call per keystroke adds up), and once the table grows past tens of thousands of rows, add a pgvector HNSW index so the neighbor search stops being a full scan.

No cloud allowed? Still works

The objection I hear most in Austrian industry projects: no data leaves the house. Fine, because the embedding model is the only external dependency here, and it’s replaceable with something that runs on a plain CPU. “Small” has gotten dramatically smaller lately, and it comes in two tiers:

The first tier is the classic route: a real transformer, quantized to int8, run through ONNX Runtime. Twenty-something megabytes, a few milliseconds per embedding on CPU, and int8 quantization keeps 95%+ similarity to the full-precision model.

The second tier is the newer, more surprising one. Model2Vec distills a sentence-transformer into static embeddings: every token’s vector is precomputed, and embedding a sentence is a lookup plus an average. No transformer inference at all, which makes it hundreds of times faster than MiniLM on CPU while its best models retain roughly 95% of MiniLM’s benchmark quality. For the per-keystroke dropdown scenario, that speed is exactly what you want, and an 8 MB model file is small enough to embed in your application as a resource.

Two honest caveats. These small models are English-centric; for the mixed German/English data common in my projects, the multilingual variants (multilingual-e5-small, or Model2Vec’s multilingual distillations) cost noticeably more megabytes and a little quality. And whatever you pick, the vector column must match the model (vector(384) for MiniLM, vector(64) for the tiniest potion model), because distances only mean something within one model’s space.

The full local pipeline

Talk is cheap, so here is the whole thing. Two packages (Microsoft.ML.OnnxRuntime and Microsoft.ML.Tokenizers) plus two files from Hugging Face (model.onnx and vocab.txt for all-MiniLM-L6-v2), and the entire generator fits in one class:

sealed class LocalEmbeddingGenerator(string modelPath, string vocabPath)
    : IEmbeddingGenerator<string, Embedding<float>> {
    private readonly InferenceSession _session = new(modelPath);
    private readonly BertTokenizer _tokenizer = BertTokenizer.Create(vocabPath);

    public Task<GeneratedEmbeddings<Embedding<float>>> GenerateAsync(
        IEnumerable<string> values,
        EmbeddingGenerationOptions? options = null,
        CancellationToken cancellationToken = default) {
        var result = new GeneratedEmbeddings<Embedding<float>>();
        foreach (var text in values)
            result.Add(new Embedding<float>(Embed(text)));
        return Task.FromResult(result);
    }

    private float[] Embed(string text) {
        // 1. text -> token ids (BertTokenizer adds [CLS]/[SEP] for us)
        var ids = _tokenizer.EncodeToIds(text).Select(id => (long)id).ToArray();

        var inputIds = new DenseTensor<long>(ids, [1, ids.Length]);
        var mask = new DenseTensor<long>(
            Enumerable.Repeat(1L, ids.Length).ToArray(), [1, ids.Length]);
        var typeIds = new DenseTensor<long>(new long[ids.Length], [1, ids.Length]);

        // 2. one forward pass -> per-token vectors [1, tokens, 384]
        using var output = _session.Run(
        [
            NamedOnnxValue.CreateFromTensor("input_ids", inputIds),
            NamedOnnxValue.CreateFromTensor("attention_mask", mask),
            NamedOnnxValue.CreateFromTensor("token_type_ids", typeIds),
        ]);
        var hidden = (DenseTensor<float>)output[0].Value;

        // 3. mean-pool the token vectors, then L2-normalize
        var dims = hidden.Dimensions[2];
        var vector = new float[dims];
        for (var t = 0; t < ids.Length; t++)
            for (var d = 0; d < dims; d++)
                vector[d] += hidden[0, t, d];

        float norm = 0;
        for (var d = 0; d < dims; d++) {
            vector[d] /= ids.Length;
            norm += vector[d] * vector[d];
        }
        norm = MathF.Sqrt(norm);
        for (var d = 0; d < dims; d++) vector[d] /= norm;

        return vector;
    }

    public object? GetService(Type serviceType, object? serviceKey = null) =>
        serviceType.IsInstanceOfType(this) ? this : null;

    public void Dispose() => _session.Dispose();
}

Three steps: tokenize, run the model once, average the per-token vectors and normalize. That averaging step is the part tutorials usually hide inside a library: the transformer produces one vector per token, and mean pooling is how all-MiniLM-L6-v2 turns them into one vector per sentence. Because the result is L2-normalized, a plain dot product is the cosine similarity.

Swapping it in is exactly the one-registration change promised above, plus the column width:

builder.Services.AddEmbeddingGenerator(_ =>
    new AzureOpenAIClient(
            new Uri("https://your-endpoint.openai.azure.com/"),
            new ApiKeyCredential(builder.Configuration["OpenAI:ApiKey"]!))
        .GetEmbeddingClient("text-embedding-3-small")
        .AsIEmbeddingGenerator());
builder.Services.AddEmbeddingGenerator(_ =>
    new LocalEmbeddingGenerator("model.onnx", "vocab.txt"));

I ran this exact class against the post’s examples, on CPU, no API in sight. And then I ran it a second time with a different model file (gte-small, another 384-dimension small model) to make a point:

dotnet run minilm
# 'Kickoff meeting notes' vs 'Project start discussion':  0.32
# 'Kickoff meeting notes' vs 'Completely different topic': 0.09

dotnet run gte
# 'Kickoff meeting notes' vs 'Project start discussion':  0.84
# 'Kickoff meeting notes' vs 'Completely different topic': 0.73

Read that carefully. Same texts, same code, same vector size, and the numbers live on different planets. MiniLM spreads its scores wide: the duplicate pair at 0.32, the unrelated one near zero. gte-small compresses everything upward: even the unrelated pair scores 0.73. Now apply the 0.2 cosine-distance gate from earlier in this post to both. On MiniLM it misses the true duplicate by a mile (its distance is 0.68). On gte-small it catches the duplicate, but the unrelated text sits at distance 0.27, one hair away from a false positive.

That is the “distances only mean something within one model” warning as a measurement instead of a sentence. Every embedding model, hosted ones included, has its own similarity geometry. The threshold is not a property of your data; it’s a property of your data under one specific model. Swap the model, re-run the observe-only log, and pick the number again.

Production notes for the honest file: this minimal version embeds one text per call (batch the tensor for bulk work), doesn’t truncate (MiniLM caps at 256 wordpieces; cut or chunk longer inputs), and should hold one InferenceSession for the process lifetime, which the DI singleton registration already gives you.

Where this stops

Embeddings are not magic similarity. They won’t catch a typo’d article number (“4597-B” vs “4579-B” are semantically close and factually different; use exact matching or trigram search for identifiers). Distances are only comparable within one model, so never mix vectors from different models in one column, and treat a model upgrade as a re-embedding migration. And every threshold is domain-specific: numbers from a blog post, including this one, are starting points for your observe-only log, not answers.

The rule of thumb: if your problem is “these two texts mean the same thing” or “find me the entries closest to what I typed”, you don’t need an AI project. You need one embedding call, one vector column in the Postgres you already run, and one ORDER BY. Everything past that is tuning.