AI Model Testing Services for LLM and RAG Systems

A test set contains between one hundred and three hundred cases taken from your own traffic, split into four groups: ordinary queries the system meets every day, edge cases at the limits of its scope, adversarial inputs written to break its instructions, and policy-sensitive inputs where a wrong answer would cost money or trust. The split is agreed with you in writing before the first run.

You remain accountable for the system you operate. A test run measures behaviour on a defined set of cases at a named version; it does not transfer liability for the system, and no report from Eicra Soft Limited says otherwise. Where your obligations come from a regulation such as the EU AI Act, the duty sits with the provider or deployer of the system, and our report is evidence you can put against that duty rather than a substitute for it.

The engagement letter names the governing law and forum, and we work to whichever your legal team requires rather than imposing ours. Eicra Soft Limited is registered in Bangladesh and delivers remotely. Where personal data is involved, the data processing agreement follows Article 28(3) of the GDPR and the transfer mechanism your own regulator requires is signed before any data moves.

Runs execute in the environment you nominate, using accounts and keys you control, with read-only access for named reviewers only. Where raw records must not leave your systems, cases can be masked or rebuilt synthetically from the same patterns. Run logs and the test set are retained for the period written into the engagement letter and deleted on your confirmation.

A starter run of 100 cases takes about two days and a standard run of 300 cases takes one to two weeks, counted from the day the rubric is agreed. A repeat run against a new version takes three to five days. The exact schedule depends on how much of the review is human and how quickly the rubric is signed off, and it is written into the scope before anything starts.

Five things happen, in this order:

1. We ask which system, which version and which use case, and put the answers in a written scope.
2. We agree the test set with you, including how the four case groups are split.
3. We agree the scoring rubric, so what counts as a failure is settled before anything is scored.
4. We run the set, log every input and output, score each output and reproduce each failure.
5. We hand over the findings report, the run logs and the test set itself.

Yes. Bring whatever exists: their test set, their rubric, their last report. We map what was covered against what was not, tell you plainly where the two disagree with our own reading, and then either extend the existing set or rebuild it. Where their set is sound we say so and reuse it rather than charging you to recreate work you already own.

No, and any vendor who says otherwise is selling something that does not exist. A test run measures behaviour on the cases it ran, at the version it ran against. It cannot promise behaviour on inputs nobody has tried, and a model or prompt change can undo a good result. That is exactly why the test set is handed to you: so the same run can be repeated after every change instead of trusted once.

Observability tells you what your system did in production; testing tells you whether it was right. Tracing dashboards record live traffic and surface latency, cost and errors after the fact. A test run puts the system in front of cases whose correct answer is already known, scores each output against a written rubric, and reproduces the failures. Most teams end up needing both.

Yes. For an agent the unit under test is the whole trajectory, not a single answer, so a case records which tools were called, in what order, with what arguments, and whether the final action was correct. Cases also cover the agent calling the wrong tool, calling nothing, or looping. Read-only access is still the rule, so cases that would write to a live system are run against a test environment you nominate.

Yes, as evidence rather than as certification. A findings report, its rubric and its run logs can be referenced against a measurement or evaluation control in a framework such as the NIST AI Risk Management Framework or ISO/IEC 42001. What we cannot do is certify conformity: Eicra Soft Limited is not a notified body or an accredited certification body, and no report says otherwise.

When nobody outside your team needs to believe the result. Open-source harnesses run the same kinds of checks and cost nothing, so if you are tuning a prompt and your own engineers are the audience, use them. The reason to pay for an independent run is that a customer, auditor or procurement reviewer will not accept a vendor marking its own work.