Skip to content
NEURIXIS

Sector

AI in a care environment

In healthcare, compliance decides the architecture before engineering does. The regulatory status of the software, where the data is hosted and the legal regime of the processing are settled at scoping. Discovered mid-project, they end it.

The context

Health data is plentiful and heterogeneous: free-text reports, activity coding, DICOM images, HL7 or FHIR feeds, and a layer of departmental spreadsheets nobody owns. Patient identifiers do not reconcile across systems, and record linkage is often the real technical subject of the project, well before any model.

Most of the clinical information sits in free text, and that text is written fast: ward-specific abbreviations, negations, hypotheses, history mixed with today’s findings. An extractor that does not explicitly handle negation and temporality turns "no evidence of" into a diagnosis. This is not an edge case, it is the leading source of error.

What matters is not average performance, it is behaviour on rare cases and the system’s ability to decline. A model that always produces an output, with the same confidence, is not usable in a care pathway. Explicit refusal and escalation to a human queue are features, not workarounds.

Segregation is real and deliberate: certified hosting, pseudonymisation, access traceability, separation between care and research purposes. These constraints are not bolted on at the end, they shape the first architecture diagram. A well-scoped healthcare project costs less than a badly scoped one restarted six months in.

Operating room fitted with advanced medical equipment.

Regulatory constraints

Three questions come before architecture: where does the data live, under which regime is it processed, and is the software a medical device? Until they are answered, there is no project.

  • HDS certification

    Hosting personal health data collected in the course of prevention, diagnosis, care or social and medico-social follow-up requires HDS certification under the French public health code. The point people systematically miss: the constraint covers training environments, development datasets and backups, not just production. An export "just to test" into an uncovered environment is a non-compliance, however temporary.

  • GDPR, Article 9

    Health data falls under the special categories of Article 9: processing is prohibited in principle, and requires both a legal basis and one of the conditions that lift the prohibition. The practical consequence is counter-intuitive: care purposes and research purposes are distinct, and data collected for one does not flow freely to the other. This is the question that decides whether you can train on your own data at all.

  • CNIL framework: research and data warehouses

    Processing health data for research relies on the CNIL reference methodologies, to which a controller can declare compliance, or otherwise on an authorisation. Building a health data warehouse falls under a dedicated framework with its own governance and transparency requirements. Choosing between these routes is the first arbitration of the project, and it drives the schedule far more reliably than the choice of model.

  • Medical Device Regulation (EU) 2017/745

    Software intended by its manufacturer for a medical purpose, diagnosis, prevention, prognosis, treatment or monitoring, is a medical device and falls under the classification and conformity assessment set out in the regulation. The line between an operational support tool and a clinical decision support tool can feel thin in conversation and never is in its consequences: it sets the cost, the schedule and the skills you need. We put it in writing at the first meeting.

  • AI Act, Annex III

    Some healthcare-related uses are high risk under Annex III, notably systems used for triage in access to emergency services. These obligations stack on top of the Medical Device Regulation where the software is also a device. The two files have neighbouring requirements on risk management and documentation, and building them together is markedly cheaper than reconstructing one after the other, once the product is already on the market.

AI systems ranked from most to least regulated: unacceptable risk, high risk, limited risk, minimal risk.Most regulatedLeast regulatedUnacceptable riskBanned in the UnionHigh riskAnnex III, heavy obligationsLimited riskTransparency obligationsMinimal riskFree use

Use cases

  • Structuring clinical reports

    The problem
    Clinical information is locked in free text. Coding, cohort building and research extracts are done by hand, on re-reading, with turnaround times that discourage use.
    The approach
    Entity and relation extraction with explicit handling of negation, uncertainty, temporality and subject (patient or relative). Language model running on HDS-certified infrastructure or on premise, never against an uncovered interface. Schema-constrained output, so it is verifiable rather than merely plausible.

    Success criterion

    Precision and recall per entity type on a set annotated by two independent clinicians, with their agreement measured. That agreement is the performance ceiling: below it nothing can be concluded, and the first genuinely useful deliverable is sometimes an improved annotation protocol.

  • Assisted internal document retrieval

    The problem
    Protocols, procedures and guidelines live across an intranet, PDFs and binders. Several versions coexist, sometimes contradictory, and the answer depends on who you ask.
    The approach
    Version-controlled indexing, hybrid lexical and vector retrieval, passage reranking, every answer cited back to the source passage and its version date, explicit refusal when the corpus does not contain the answer, and logging of questions to revise the corpus.

    Success criterion

    Share of correctly sourced answers and share of well-founded refusals, on a question set written by the professionals themselves. Plus the rate of claims not backed by a cited passage: that figure has to be zero on the qualification set before go-live, and it is not negotiable.

  • Activity and workload forecasting

    The problem
    Bed, theatre and staffing planning runs on history and intuition. Overstaffed one day, stretched the next, with nobody able to say in advance which is coming.
    The approach
    Time series with multiple seasonalities, hour, day, week, school holidays, epidemic season, producing a prediction interval rather than a point estimate. Staffing decisions are taken on the upper bound, not the mean. This is the point most dashboards get wrong.

    Success criterion

    Absolute error per slot on a period after the training window, against a seasonal naive baseline and the current plan. Plus empirical coverage of the intervals: a stated 90 % interval must contain the observed value nine times out of ten, otherwise it is decoration.

  • Coding and billing consistency checks

    The problem
    Coding feeds billing and activity reporting. Discrepancies surface late, at audit, when correction is most expensive and consumes already scarce clinical time.
    The approach
    Classification on historical coding combined with explicit business rules, positioned as discrepancy detection and never as automatic coding. The output is a ranked queue of files to review, with the elements that triggered the flag and the estimated amount at stake.

    Success criterion

    Precision at the top of the queue: the share of flagged files where a reviewer confirms a real discrepancy, measured over the first files in the queue at constant review capacity. It is the only number the department cares about, and the one we commit to producing.

A two-by-two matrix crossing integration effort with estimated annual gain, with four named quadrants.Quick winHigh gain, low effortStructural projectHigh gain, high effortGadgetLow gain, low effortTrapLow gain, high effortEstimated annual gainhighlowIntegration effortlowhigh

What makes projects fail here

Healthcare AI projects rarely fail on the model. They fail on scoping, on data access, and on the clinician’s desktop.

  • The project starts without the regulatory status of the target software written down. Six months later, the chosen intended purpose makes it a medical device. Schedule and budget no longer resemble what was presented.
  • A data extract leaves the organisation "just to test" into an uncovered environment. The project stops, and it is right to stop. This is handled up front with the DPO and the CISO, never afterwards with apologies.
  • The model is evaluated on a set drawn from the same ward, the same period and the same source system as the training set. It does not hold on the ward next door, and nobody tested it on the ward next door.
  • No clinician co-wrote the evaluation protocol. The model then optimises a technically clean criterion that the service does not recognise as a measure of its problem.
  • The tool is good but it adds two clicks in software where time per patient is already squeezed. It goes unused. Integration into the existing desktop decides adoption, and it is designed at the start.
  • Pseudonymisation is treated as swapping identifiers in a database, while the free text carries names, dates, addresses and clinician names. It has to be handled inside the text, then verified on a manually reviewed sample.
The closed MLOps loop: train, deploy, monitor, detect drift, retrain.TrainDeployMonitorDetect driftRetrain

What we do, and what we do not do

What we do

  • We bring in the DPO, the CISO, IT and clinicians at the first meeting, before any architecture diagram.
  • We put the regulatory status of the target software and the applicable processing regime in writing, and we scope accordingly.
  • We design on HDS-certified hosting or on your infrastructure, training environments included.
  • We handle pseudonymisation inside the free text, verified on a manually reviewed sample, not just in identifier columns.
  • We co-write evaluation protocols with clinicians and measure inter-annotator agreement before quoting any performance figure.
  • We produce the AI Act documentation and the technical inputs to your compliance file.

What we do not do

  • We are not a health data host, not a certification body and not a notified body. We design for these frameworks, we do not issue them.
  • We give no medical advice and we do not validate clinical performance on your behalf. That responsibility stays with your clinicians and your organisation.
  • We do not take projects where the data cannot stay inside a compliant environment. There is no acceptable workaround on this point.
  • We do not replace your clinical software vendor and we do not take over maintenance of their product.
  • There are two of us. We do not run parallel rollouts across multiple sites, and we would rather say so before the proposal.

Frequently asked questions

  • Does our health data stay on HDS-certified hosting?

    Yes, including for training. HDS certification (the French standard for hosting personal health data) does not cover production alone: it also covers training environments, development datasets and backups. An export "just to test" to an uncovered environment is a non-compliance, even a temporary one. We design on HDS-certified hosting or on your own infrastructure, and we do not take on a project where the data cannot stay in a compliant environment.

  • Will your software be a medical device?

    It depends on the intended purpose you give it, and we qualify that in writing at the first meeting. Software intended for a medical purpose, diagnosis, prevention, prognosis, treatment or monitoring, falls under the medical devices regulation and its conformity assessment. The line between an organisational aid and a clinical decision aid can be thin in conversation, never in its consequences: it decides the cost, the timeline and the skills to bring together.

  • How do you handle pseudonymisation in clinical notes?

    In the text itself, not just in the identifier columns. A free-text note contains names, dates, addresses and practitioner names: treating pseudonymisation as a simple swap of identifiers in a table lets everything else leak. We handle it inside the text, then verify it on a re-read sample, because pseudonymisation that has not been checked is only a hypothesis.

  • Can we train a model on our care data?

    It depends on the legal basis for the processing, and it is the project’s first decision. Health data falls under the special categories of Article 9 of the GDPR: processing it requires a legal basis and one of the conditions that lift the default prohibition. A care purpose and a research purpose are not interchangeable, and data collected for one does not move freely to the other. This question decides whether training on your own data is feasible, and it is settled with your DPO before any architecture diagram.

Let us talk about your use case.

An hour is enough to tell whether your subject stands up. If it does not, we will tell you then, not after three months of project.