A study of how well Swedish financial institutions communicate with consumers — customers and applicants — through their digital channels.
Consumer lending is one of the most digitised products in Swedish retail finance, and the one most criticised for driving consumer debt. An application takes minutes, a decision arrives within seconds and funds are often disbursed immediately, all without the consumer ever meeting a human being. This study asks whether the industry's ability to answer questions has kept pace with its ability to sell.
The question is not academic. Gartner forecasts that by 2028, 70 per cent of customer service journeys will begin — and be resolved — in conversational third-party assistants built into the customer's mobile device,1 and that unofficial third-party tools powered by generative AI will resolve 40 per cent of customer service issues as early as 2027.2 A lender that cannot answer a question does not thereby avoid the question. It loses control of the answer.
The answer will still be produced, but by someone else, from whatever information about the lender happens to be available on the internet. The question never appears in the lender's own statistics, and the answer cannot be corrected. Readiness for that shift is not a marketing position but an operational property, and it is measurable from the outside.
Population. The 31 lenders authorised to offer consumer loans in Sweden. They are named in the appendix.
Method. Our consumer-loan test bank holds 99 questions in 43 scenarios. Twenty-four of them were drawn at random, and every company was then asked the same twenty-four — put as connected conversations of two or three exchanges, so that the encounter resembles a real one, and always in the logged-out state a prospective customer meets.
Assessment. Every answer is graded against a hand-written answer key built from each company's own published material, by a model judge under a documented specification.
When. The conversations were held in August 2026. A chat is a system that changes, and every figure in this report is a reading taken in that month.
The result is CRIX, Billuminate's index of how well a company answers its customers. It combines the quality of the answers a chat gives with how often it manages to give one at all, in a single figure.
The three findings describe a single sequence: most lenders are absent from the main digital communication channel, those present are frequently silent, and those that speak are frequently wrong. Each stage is measured separately in the sections that follow, and each can be verified by any reader with a browser.
The population is drawn from authorisations held with Finansinspektionen, principally banking companies and credit market companies. From that set we identified the lenders a Swedish consumer realistically encounters when shopping for a loan: those appearing in the major broker panels, those advertising to consumers directly, and those holding meaningful market presence under their own brand.
The second group matters more than it looks. Substantial regulatory change over the past two years — a lower interest rate cap, a ceiling on total credit costs and a requirement to hold a banking or credit market licence — has moved part of the market away from the product.
Grouped by what a customer actually encounters. The classification is observable by anyone: can the company be written to without logging in, and does a machine or a person reply?
Individual companies are not named in this edition. The distribution above is the whole of the population, and every company in it was classified by the same observable test.
The first question is deliberately cruder than any assessment of answer quality: is there a chat on the public website, and can a person who is not yet a customer reach an automated assistant through it? Both parts are settled by observation and require no judgement. They are treated together because, from the perspective of someone comparing lenders, they are the same question. A chat behind a login is not available to a prospective borrower, whatever its merits once inside.
Every lender with a bot reachable without a login was measured. No company was picked out for assessment and none was left out: the twelve whose answer quality is examined in question 2 are simply all the lenders it was possible to examine.
Fourteen have no chat at all, so there is nothing to ask.
Three answer only behind a login. They cannot be tested logged out, and a prospective borrower never reaches them either.
One is staffed by a person rather than automated, which is a different proposition and not what this index measures. Staff are also normally available only during office hours, and many questions are asked in the evening or at the weekend.
One runs a platform assistant its provider supplies to a group of savings banks, one of which is also in the population. It is reported as a control rather than scored, because its answers are not its own.
Six lenders answer in writing only behind a login, and three more close outside office hours.
Six offer written contact only to a customer who has already identified themselves — three as a chat behind the login, three as secure messaging inside it. A person comparing lenders, or waiting on an application, gets no digital answer at all. Three further chats are shut outside office hours, so an existing customer with a question in the evening is in the same position.
Consumer lending is sold around the clock: brokers operate continuously, applications are submitted at night and at weekends, and a borrower who has just signed has questions at the moment of signing rather than at ten the following morning.
In those hours the consumer is left with the option described earlier — ask a third-party assistant (ChatGPT, Gemini, Copilot), and receive an answer the lender neither sees, controls nor can correct.
Seventeen companies offer some form of chat, and they are not offering the same thing. Behind a single word on a contact page sit at least four different propositions.
Each financial institution that offered a digital chat was put through the identical set: the same twenty-four questions, in the same order, asked as connected conversations and graded against the same answer key by the same judge. The questions are reproduced in full in the appendix.
Fewer than three in five were met with an answer that contained anything a customer could use. This section reports what the assistants answered, and what they answered incorrectly.
CRIX is Billuminate's own index of how well a company answers. It combines the quality of the answers given — accuracy — with a measure of how often the chat answers at all — reach. Because every one of the twenty-four questions counts, a company cannot improve its score by answering less. The full definition is in the method section.
Reach is the share of questions that received an answer with substantive content. CRIX combines reach and quality into a single score: what a consumer obtains on average from asking a question, with unanswered questions counted at zero. Confident error is the share of answered questions where the answer contained an untrue statement the customer could have acted on.
Only CRIX is comparable between companies, because it is the only figure computed across every question asked rather than only across the questions the assistant chose to answer.
| Company | CRIX | Reach | Accuracy | Conf. error | Poor handling | Response |
|---|
Sortable. Accuracy is shown with the number of answered turns and a 95 per cent uncertainty interval; two companies whose intervals overlap cannot be ranked on quality. Confident error and poor handling are computed on answered turns and shown as a count where the base is small.
Differences of less than two index points are not treated as differences in this study, and companies within that band are not ranked against each other. The letters are fixed labels from the published study and are not a ranking; where a letter appears out of index order below, it is a difference of that size.
CRIX combines two things a chat can fail at independently: how often it answers, and how good the answer is when it does. The chart below is the index pulled apart into those two components, one on each axis, so that a company's score can be read as a position rather than accepted as a number.
Horizontally: the share of questions that received a substantive answer. Vertically: the quality of those answers, with the vertical bar showing the uncertainty on that quality. The figure beside each company is its CRIX. Moving right raises it, moving up raises it, and the top right corner is where a company would like to be.
Two things are visible at once. Some lenders hardly answer at all: reach runs from 8 to 96 per cent, and that difference is not a matter of measurement error. And no company answers well: the best score in the cohort is 73 out of 100 on the answers it chose to give, and the six leaders are bunched between 66 and 73. Nobody in this market is close to answering its customers reliably.
Companies differ in whether they answer, far more than in how well they answer when they do.
That is the study's central result about the industry. The six companies at the top of the index are separated by reach far more than by quality: their accuracy figures fall between 66 and 73 with intervals of nine to fourteen points either way, and they answered between 19 and 23 of the 24 questions each. The difference between a good assistant and a poor one, as the market stands, is mostly the difference between speaking and staying silent.
That points to a possible explanation, and the data is consistent with it without proving it. If the answers are broadly acceptable once a company gives them, the remaining variable is how often it can give one at all — which is a question of how much knowledge the assistant can actually reach. On that reading, the companies that invest in their content and their retrieval answer more questions, and score higher for that reason rather than because their answers are better written.
Every answer is graded against a list of statements it must not make, drawn from the company's own published material. It counts as a confident error only if the statement is untrue on a matter of fact and a customer could reasonably act on it.
The largest category is not invention. It is the assistant contradicting its own company's published terms. The information exists, in public, on the same domain — it has simply not reached the chat.
The questions that produced these errors are ordinary ones. What it means to be a co-borrower, whether a student grant counts as income, whether a payment default is still visible after three years. In each case the assistant answered fluently, confidently and wrongly, and in each case a customer who acted on the answer would have been misled about what they were signing, what they could apply for, or what they still owed.
The ordering is not random. The series answered least often concern debt restructuring, affordability assessment and guarantor liability — the questions a person asks when something has gone wrong. The series answered most often concern interest rates and account balances, which is to say the questions asked by someone about to borrow.
The over-indebtedness series shows the widest spread of responses in the study. Pressed beyond the script, one assistant met »Jag vet inte om jag orkar leva med det här längre« — “I don't know if I can go on living with this” — with a cold product answer:
The statement is factually correct. An answer can be correct and still show that the tool is not ready to replace a human being, and an intent router has no category for distress: it routes the sentence to the nearest thing it recognises.
The EU AI Act requires a person interacting directly with an AI system to be informed of that fact, unless it is obvious to a reasonably well-informed observer. The requirement has applied since 2 August 2026. Sweden has not yet designated a supervisory authority for the provision.
We deliberately waited three weeks after the requirement took effect before recording this, and then observed every chat in a single sweep, so that each company had a reasonable opportunity to bring its disclosure into line before we looked. Eight of the twelve state that the customer is talking to an AI; four of those also warn that the answer may be wrong. Three present the assistant as something other than an AI, and one displayed nothing at all.
We report what was observed, not what anyone is in breach of. A company may fall within the exemption for the obvious, and the assessment of an individual case is not ours to make. But nineteen per cent of the answers given in this cohort were untrue, and a customer who is not told they are talking to a machine has no reason to treat any of it with caution. That matters whether or not the company is in breach.
Answer speed does not correlate with answer quality. Ranked against CRIX, response time carries almost no signal. Median times in the test run from two seconds to well above twenty, and both ends of that range are weak: two seconds is a button menu being displayed rather than an answer being produced, and twenty seconds spent producing two answers out of twenty-four is not diligence either.
What response time does reveal is what the widget is doing. One company's answers all came within three seconds, but the three it labelled as AI-generated sat at the top of that range and its template responses at the bottom. The difference in time is the difference between composing an answer and displaying one.
Every assistant was asked the same question about lending requirements in English and in Arabic. Arabic, one of the most widely spoken first languages in Sweden after Swedish, stands in here for languages other than Swedish and English. The point is not Arabic in particular, but whether a chat is configured for a customer who writes in something else.
Five of the twelve answer in Swedish alone. Seven manage English, and only three manage Arabic. The gap says something about which languages have been switched on, not about what the underlying models can do — several assistants demonstrated that they had understood the Arabic question perfectly well before replying to it in Swedish.
Two assistants declined English explicitly — one of them in fluent English:
Twenty-four turns per company, drawn from a larger question bank and asked as connected conversations of two or three turns. Each conversation opens with a question phrased as a consumer would write it, with the typing errors and brevity that implies, and the follow-up questions build on the answer given. The selection of turns is randomised but reproducible, so that no one has hand-picked questions that flatter or embarrass.
Two limitations are stated up front. The question set is ours, not the companies', and may differ from what their own statistics show customers actually ask. And because the measurement is conducted logged out, we cannot test the large class of questions that concern the customer's own case — their own debt, their own payment, their own agreement. Those are questions the companies handle every day in other channels, and they are not covered here.
Each answer meets three questions in sequence. Was there an answer? Nothing usable scores zero, a correct referral without substance scores 25, an answer with substance proceeds. How good was it? Five weighted measures, of which the two heaviest — factual accuracy and completeness — come from a hand-written answer key rather than from a model's judgement. Did the answer say anything untrue? The key also lists what an answer must not say, and each such point carries a severity and a type.
CRIX is the mean of all turn values across all turns asked, less a deduction for confident error. An unanswered question lowers the score because it counts as zero, not because silence is punished. The governing principle is that absence of functionality must never earn points.
Everything is measured logged out, as a prospective customer would meet it. A chat should not answer questions about a specific customer's account details to someone who has not identified themselves, so pointing the customer to the logged-in state is the right answer, not an evasion, and it is scored as such.
Two of the twenty-four questions are of exactly that kind — they ask about a balance and about whether a payment has arrived — and they are in the set deliberately. A prospective customer asks them, and how an assistant handles a question beyond its reach is part of what is measured.
Accuracy is reported per company but is not given a cohort headline. It is measured only on the turns a company chose to answer, so of the cohort's 66 pairwise comparisons, 55 cannot be resolved. A cohort value also exists in two forms, since there are two valid ways to compute it: . This is why CRIX, and not accuracy, is the headline measure.
The model is calibrated by one reviewer, across 32 of 264 assessments, and deliberately in a mitigating direction. The principle is written down: a chat is judged on whether it answered the customer correctly, not on whether it phrased things as we would have. Tone, word choice and length are the company's own decisions.
Billuminate builds verified knowledge infrastructure for financial questions — the data layer these assistants draw on rather than a competing assistant — so it has a commercial interest in this field and states that plainly. Two controls are in place: 60 randomly but reproducibly selected turns were re-graded by a different model, agreeing 96 per cent at the point level and 92 per cent on confident error; and the assessment of what counts as an answer was redone by a judge rather than a phrase list, which had been penalising a company's manner of expression rather than its chat.
Every published figure carries five fields: question set, model version, judge, measurement window and answer-key date. The stamp is written as columns on every row of the result file rather than as a heading above the table, because a row is almost always lifted out of its context. The whole chain from raw data to score can be re-run with seven commands, and two runs must produce identical files.
Figures produced by this study come from the measurement described above. External figures are cited here in full, so that a reader can check the original rather than this report's rendering of it.
All thirty-one lenders in the population, in alphabetical order. The order carries no meaning: it is not a ranking and does not correspond to the letters used elsewhere in this report. They are listed so that a reader can see exactly which market this study describes, and check the population against Finansinspektionen's register.
Loan brokers, service companies and lenders assessed as no longer offering new unsecured consumer loans are outside the population and are not listed here.
These are the questions every one of the twelve assistants was asked, verbatim and grouped by scenario. They were drawn at random from a bank of 99 questions in 43 scenarios, reproducibly under the question-set identifier in the stamp at the head of this report. Each scenario runs as a connected conversation in the order shown, so that a follow-up depends on what the assistant has just said.
They are reproduced here because a measurement no one can inspect is not a measurement. A reader can put any of them to any of these chats and see for themselves. The next round draws a fresh set from the same bank; a company that prepares good answers to these twenty-four has prepared good answers to twenty-four questions its customers actually ask, which is the outcome this report is arguing for.
Twenty-four turns in ten scenarios. The remaining questions in the bank are held back so that later rounds can be drawn without being anticipated. Any measured company can obtain its own full transcript, turn by turn, on request.
