Blog

Today was all about making pragmatic technical decisions and getting my hands dirty with local LLMs.

Two major milestones: finalizing my database choice and successfully running a local model for data extraction.

Part 1: The Database Decision

Discovering NoSQL Varieties

I’ve only ever worked with Firebase Firestore (document-based), so learning about the different types of NoSQL databases was eye-opening. AWS has a great overview of the various types—key-value, document, graph, and more. Each has its place depending on the use case.

This reinforced something I truly believe: in tech, nothing is inherently good, bad, or best. It all boils down to the use case.

MongoDB Atlas vs Self-Hosted

After learning some system design principles, my initial instinct was to go with self-hosted MongoDB. It’s free, open-source, and gives me complete control. However, for V1 of HuntKit, MongoDB Atlas’s free tier (M0) makes more sense.

Why Atlas wins for now:

The M0 free tier limitations are actually quite manageable for my use case:

• 100 CRUD operations/second – I’m building a crawler that needs intentional delays anyway. Even 50-60 operations would suffice, and a future caching layer will reduce this further.
• 512 MiB storage – More than enough for the scale I’m targeting initially.
• 500 concurrent connections – This isn’t a user-facing database. I’ll rarely have more than 1 concurrent connection. Connection pooling becomes critical later with the SQL database for user data.
• No backups – A risk, yes, but acceptable for V1.

I found a solid article comparing Atlas to Community Edition that shows the costs are nearly identical at scale, with Atlas offering significantly more convenience. The MongoDB dependence is a trade-off, but it’s the right choice for getting to market quickly.

GitHub repo initialized: https://github.com/maitry4/HuntKit (just testing for now, no code yet)

Part 2: Local LLM Adventures

Today’s main challenge: get a working local LLM running for data extraction.

The DeepSeek Reality Check

I started following a Dev.to tutorial on running DeepSeek locally using Docker.

Big problem: DeepSeek V3 is impossibly large.

Yesterday I kept seeing conflicting information—400+ GB to 4GB. Today, when I actually tried to build it, reality hit. I can’t even download it on my machine, let alone run it efficiently.

Finding the Right Model

After some research, I pivoted to smaller models:

• Llama models – Too small and not reliable enough for my needs
• Phi-3 Mini & Qwen 2.5 3B – These look promising for future phases

I settled on Phi-3 Mini for today’s testing. At 2.2GB, it’s realistic for local development.

Architecture Insight

Before jumping into implementation, I had an important realization: I need to clean up the HTML before passing it to the model. These smaller models won’t handle raw, messy HTML well. This preprocessing step is critical.

What I Built Today

Setup Steps:

1. Installed Ollama from ollama.com
2. Pulled phi3:mini model (ollama pull phi3:mini)
3. Tested it in the terminal
4. Tested with my custom extraction prompt

The Prompt: I crafted a strict prompt that enforces JSON output without any markdown wrapping or explanations. The model must extract job information (Role, Salary, Deadline, Link) in a specific format.

Test Case: I tested it on an actual Amazon job posting (SDE II role, Job ID 3144589). Used the raw scraped HTML with some manual tweaks. It worked!

Key Learnings

• Model choice depends on constraints (storage, memory, speed)
• Preprocessing is essential for smaller models
• Terminal testing validates the approach before building the full pipeline

Tomorrow’s Plan

HuntKit Adaptations:

1. Replace terminal interaction with Python batch script calling Ollama API
2. Test the exact prompt on 10 different career pages

Future Evolution: Once validated, migrate to Docker + vLLM for faster inference at scale. Today’s setup is UI-focused; production needs optimization.

Reflections

Today reinforced the importance of starting with realistic constraints. Could I have pushed for DeepSeek? Maybe with cloud GPUs.

But phi3:mini gets me moving today, and that’s what matters for V1.

Sometimes the best technical decision isn’t the most impressive one—it’s the one that ships.