A Python library for parallel LLM inference across providers, built on Polars DataFrames.
Polar Llama is a Python library that enables parallel inference calls to multiple Large Language Model providers through Polars dataframes. It streamlines batch processing of AI queries without serial request delays, making it ideal for data-intensive AI applications.
Send multiple inference requests in parallel without waiting for individual completions
Leverages efficient Polars dataframe operations for request management
Supports context-preserving conversations across multiple message exchanges
Connects with OpenAI, Anthropic, Gemini, Groq, and AWS Bedrock models
Classify documents with custom taxonomies including detailed reasoning, reflection, and confidence scores
Native support for structured outputs with Pydantic models and JSON schema validation
Get started with a simple example
Classify documents according to a custom taxonomy with detailed reasoning and confidence scores
Taxonomy-based tagging is a powerful feature that allows you to classify documents according to a custom taxonomy with detailed reasoning, reflection, and confidence scores. This feature is particularly useful for content classification, customer support routing, email triage, sentiment analysis, and multi-label classification.
For each possible value in each field, the model provides its reasoning
After considering all options, the model reflects on its analysis
Each classification includes a confidence score (0.0 to 1.0)
Multiple documents and fields are processed in parallel automatically
Get started with taxonomy tagging in just a few lines of code
Learn how to create effective taxonomy definitions
A taxonomy is defined as a dictionary with the following structure:
Understanding the structured output format
Each tagged document returns a Struct with the following nested structure:
| Field | Description |
|---|---|
| thinking | A dictionary with reasoning for each possible value in the taxonomy |
| reflection | The model's overall reflection after considering all options |
| value | The selected value (one of the values from the taxonomy) |
| confidence | How confident the model is in its selection (0.0 = not confident, 1.0 = very confident) |
How to extract and work with taxonomy tags
Advanced patterns for filtering and aggregation
Real-world applications of taxonomy-based tagging
tag_taxonomy() function signature and parameters
| Parameter | Description |
|---|---|
| expr | The document expression to analyze and tag |
| taxonomy | Dictionary defining the taxonomy structure |
| provider | The LLM provider to use (OpenAI, Anthropic, Gemini, Groq, Bedrock) |
| model | The specific model name to use |
| Returns | Polars Expression with structured tags as a Struct column |
Tips for effective taxonomy-based tagging
Begin with 2-3 fields and expand as needed
Verify that your value definitions are clear and distinguishable
Filter or flag low-confidence results for review
Spot-check classifications to ensure quality
Refine your taxonomy based on results
Always check for and handle error cases
Get type-safe, validated responses from LLMs in a predictable format
Structured outputs allow you to define the exact schema you want the LLM to follow, ensuring responses are properly formatted and can be directly used in your data pipelines. This is perfect for extracting specific information, generating consistent data, or integrating LLM outputs with databases and APIs.
Define your output schema with Pydantic models for guaranteed type correctness
Automatic validation ensures responses match your schema before processing
Get predictable, parseable outputs across all your inference requests
Seamlessly integrate with databases, APIs, and data processing pipelines
Define a simple schema and get structured responses
Maintain context across multiple messages for more natural interactions
Process customer feedback at scale
Polar Llama supports multiple LLM providers
Set up your API keys in a .env file:
Run tests with configured providers:
Process large datasets with AI insights - sentiment analysis, classification, entity extraction with validated structured outputs
Generate product descriptions, marketing copy, or documentation at scale with consistent formatting
Summarize documents, extract key points with structured metadata, or answer questions about large text corpora
Automate repetitive AI tasks like code review, email categorization, or data enrichment with type-safe outputs
Licensed under MIT.
Questions or issues? Open one on GitHub.