Litigant Assistant
System / Product: Litigant Portal
Model Type: Prompted language model with tools
Version: 1.0
Date: 2026-11-18
Owner: Free Law Project, individual owner pending confirmation
Status: Draft. Live deployment and model selection await confirmation.
1. Purpose
1.1 What this model does
A person handling a case can ask about a court procedure and get a plain-language reply. For example, for a North Dakota name-change fee question, the assistant is instructed to load the relevant court guide, find the fee in the supplied material, and cite the supporting passage. If the material does not answer the question, it should explain the gap. Assistant instructions · Court guide loader
1.2 Intended users
Self-represented litigants read the replies. Chat can also propose structured answers used by the portal’s guided forms. The litigant can inspect values and supporting text, correct them, and confirm them before form prefill. Fact recording · Review and confirmation interface
1.3 Scope and limitations
The supplied court content determines coverage. The repository includes North Dakota adult name change, Franklin County, Ohio eviction, and Arizona civil traffic. Availability depends on the court and enabled flows. Personal legal strategy, outcome predictions, and claims that a filing has occurred are outside the assistant’s role. Court content · Assistant boundaries
2. Base Model
| Field | Details |
|---|---|
| Base model name | Site-selected assistant family. The default prefers an available Balanced model, then an Economy model. The documented first choice is GPT Terra. |
| Version / checkpoint | Each site resolves a release from its Bedrock catalog. The cited benchmark used us.openai.gpt-6-luna through Bedrock Converse in us-east-1 for the comparison below. The deployed release is unverified. |
| Provider | Amazon Bedrock hosts inference. The model developer depends on the resolved family. |
| License | Model-specific terms. Confirm the deployed model and applicable terms with the owner. |
| Model location | Public reference for the deployed model awaits confirmation. Model discovery and invocation |
2.1 Model Modifications
Prompting only. FLP uses provider-hosted weights and controls the instructions sent with each request.
3. Data
3.1 Pretraining Dataset
FLP used the hosted model as supplied. No FLP pretraining dataset applies.
3.2 Finetuning Dataset
No FLP finetuning dataset applies.
3.3 Prompt Design and Versioning
The shared prompt combines behavioral rules, evidence requirements, court contacts, and available flows. Calling a tool supplies the selected flow in the conversation. The engine records the shared prompt and tool schemas by content hash, and stores identity-specific instructions with the individual message. These records let authorized reviewers reconstruct a turn when code or court content changes. Prompt construction · Transcript audit guide
3.4 Validation and Test Dataset
The November 9 baseline used 15 North Dakota reference questions, six fictional cases, and four multi-turn conversations. The frozen court material is a repository snapshot awaiting court review. Several scenarios derive from development QA scripts. Independent holdout status, provider-training overlap, and representativeness of portal traffic are unverified. Case set · Reference provenance · Holdout discussion
3.5 Label Documentation
No FLP training labels apply.
4. Training & Evaluation
4.1 Design Decisions and Rationale
A court guide supplies passages with stable identifiers, so a reviewer can trace a claim back to its source. The application logs unfamiliar citation IDs. A known ID can still point to a passage that fails to support the claim. The assistant can save an answer as unconfirmed, while only a person can confirm it. Citation checks · Confirmation boundary tests
4.2 Metrics and Evaluation
The table reports mean gated score out of 100 on the 15 North Dakota reference questions, three repetitions each. All 45 attempts per row received a grade. GPT Luna generated every answer. The external GPT-6.1 Sol grader compared answers to the frozen references. An answer with a critical failure receives an overall score of zero. The settings for generation timeouts and retries also change when runtime review is enabled.
| Configuration | Graded / planned | Mean score | Answers with a critical failure |
|---|---|---|---|
| Raw model, no court tools or runtime judge | 45 / 45 | 6.0 | 42 |
| Portal with court tools, runtime judge off | 45 / 45 | 81.7 | 6 |
| Portal with court tools and GPT Luna runtime judge | 45 / 45 | 83.3 | 5 |
Source: saved run 20261109_204106-frozen-corpus-baseline, grading batch 20261110_151211-43d56f, real-reference group. These are results for one candidate on the named questions. The benchmark does not establish legal accuracy across courts. Benchmark evaluator · Evaluation protocol
4.3 Failure Analysis
In that 45-answer group, the external grader flagged a missed required referral in six answers with runtime review off and five with Luna review on. A plausible procedural response can still omit the help the scenario calls for. The rubric also checks unsupported legal claims, incorrect facts, and fabricated or misapplied citations. Grading rubric
5. Known Limitations
Performance for other jurisdictions, languages, literacy levels, and assistive communication styles has not been established by this baseline. A missing or stale guide limits the answer even when the model follows the prompt. A user-confirmed fact records the user’s review of a value. It does not independently establish that value’s legal significance.
6. Deployment and Monitoring
6.1 Deployment Setup and Data Dependencies
The Django chat service sends the conversation and enabled tool descriptions through LiteLLM to Bedrock. It stores replies and tool activity with the thread. The portal can read attachments and optionally review a draft before release. A stream failure offers a guided page when available. Chat service · Document reader · Reply review judge
6.2 Monitoring Plan
The engine records model usage, unknown citation IDs, and stream failures. Authorized staff can review the prompt, sources, tool activity, and outcome in a transcript. The checked-in audit path provides a way to investigate a reported answer. Production review ownership, sampling frequency, and alert thresholds await confirmation. Audit workflow
6.3 Re-evaluation Criteria and Process
Proposed triggers are a new model release, revised prompt or court flow, a change in fact handling, or a report of unsupported guidance. Replay relevant cases against the same frozen references for comparison. Have a court-content reviewer examine changes to substantive claims before interpreting a score change as an improvement.
7. Version History
| Card version | Date | Author | Change |
|---|---|---|---|
| 0.1 | 2026-11-18 | Codex, prepared for FLP review | Initial draft against portal revision b20fec4. |
8. Output Disclosure
Chat replies and inferred facts can influence a litigant’s actions. The chat styles user and assistant messages differently. On an ordinary reply, the message bubble and built-in Copy action supply no explicit AI-origin label. A reply flagged during runtime review carries an accuracy warning, which Copy preserves. The form review card distinguishes “Not confirmed yet” from “Confirmed” and can display supporting evidence. That identifies review status, while a detached reply or value can still lose its origin. A proposed improvement is to identify AI-written replies beside the message and in copied text, and identify suggested values before confirmation. Chat interface · Copy behavior · Fact review card
9. Contacts
| Role | Name | Contact |
|---|---|---|
| Feature owner | Litigant Portal team. | Public contact pending confirmation. |
| AI team contact | John Gardner | john@free.law |