Almost every software category now has products described as AI-powered. Sometimes that represents a substantial change in what the software can do. Sometimes an existing product has simply acquired a conversational interface.
The distinction matters more when the software has access to financial accounts, bank data or customer records.
A useful way to evaluate these products is to start with a simple question: Can the output be checked at the moment someone needs to act on it?
That separates two very different types of claims.
Separate calculations from predictions
Consider a few statements a financial tool might produce:
“These three funds have 41 percent weighted overlap.”
“Your largest position represents 22 percent of your portfolio.”
“This stock returned 9 percent during a period when its benchmark returned 14 percent.”
These are calculations. The underlying data can be retrieved, the arithmetic repeated and the result confirmed.
Now consider another set:
“This asset will outperform next quarter.”
“This is the optimal portfolio allocation.”
“This stock is likely to rise.”
These are predictions. There is no way to verify them when the decision is being made. Their accuracy can only be evaluated later, and even then a single correct prediction does not establish whether the result came from skill or chance.
That does not make predictions inherently useless. It means they require a different standard of evidence.
The problem is that software can present both types of output in exactly the same interface and with exactly the same confidence. Users therefore need to distinguish between something the software has calculated from available information and something it believes may happen next.
Once that distinction is clear, several other questions become easier to ask.
1. What can the software actually access?
The authorization screen is usually more informative than the marketing page.
If a product exists to analyze information, consider why it would need permission to modify that information. A portfolio analysis product, for example, does not necessarily need authority to trade securities or transfer money simply to calculate concentration or performance.
The same principle applies outside finance. Software analyzing customer records does not automatically need permission to modify them. A tool summarizing an inbox does not necessarily need permission to send email.
There is a meaningful difference between a product promising not to use a capability and a product never receiving that capability in the first place.
The latter provides a structural limitation on what can go wrong.
2. How would anyone know when it is wrong?
Obvious failures are generally easier to manage than plausible ones.
If software crashes or reports that it cannot complete a calculation, the user knows something went wrong. A more difficult problem is a result that looks reasonable, is presented confidently and happens to be incorrect.
That is particularly important with AI interfaces because fluent explanations can make incorrect outputs look more convincing than an ordinary software error.
Before relying on a tool, it is worth asking how an important result could be independently checked.
The answer should not necessarily require reproducing the product’s entire job. But there should be some way to trace important calculations back to source data, inspect assumptions or understand how the result was produced.
If a consequential number cannot be checked at all, it should be treated differently from one that can.
3. What information is missing?
Every data connection has limits.
A financial application might receive current positions and balances but lack complete historical transaction information or cost basis. That restricts what the application can calculate accurately.
A product with incomplete cost-basis data, for example, can still calculate how a security performed over a defined market window. It may not be able to accurately calculate an individual investor’s lifetime profit or loss on that position.
The distinction is useful because it reveals how a product behaves when its data is incomplete.
Reliable software should identify those boundaries. If the data cannot support a calculation, the product should either say so or calculate something narrower that the available data does support.
A precise-looking answer is not necessarily a well-supported answer.
4. Where does the information go?
Financial and customer data can pass through several systems before an answer appears on screen.
Users evaluating a product should know where that information is stored, which subprocessors can access it, how long it is retained and whether it can be used to train models.
Generic statements about security are not substitutes for those answers.
A company can have strong encryption and still have a data policy a customer would not accept. Security controls and data-use policies answer different questions.
For sensitive applications, both matter.
5. What is the product responsible for?
Financial products make this question particularly important.
Software can provide information, perform calculations, generate research or facilitate actions without necessarily providing regulated investment advice. Users should be able to understand which role a product is claiming.
A disclaimer by itself does not settle every regulatory question. But a product should at least describe its role consistently in its interface, documentation and actual behavior.
The more consequential the action, the more important this becomes.
Information that helps someone understand a portfolio is different from software making investment decisions on that person’s behalf. Products should not rely on a conversational interface to make that distinction disappear.
6. Remove the AI and look at what remains
Another useful test is to imagine the product without its chat interface.
What is underneath it?
There might be a valuable data connection, portfolio-analysis engine, workflow system or calculation layer. In that case, AI may simply make an already useful system easier to operate.
In other cases, removing the conversational interface reveals very little underneath.
Neither automatically makes a product good or bad. But the exercise helps separate the value created by the underlying system from the novelty of interacting with it through natural language.
For products handling important data, the underlying capabilities generally matter more.
What this looks like in investing software
Walnut provides one example of how an investing product can approach these trade-offs.
The product connects to an investor’s existing brokerage account and is designed primarily around portfolio analysis rather than forecasting. Its analysis includes measures such as concentration, look-through overlap and benchmark-relative performance, which can be checked against the underlying portfolio data.
Walnut’s website provides more detail about how the product works.
Its approach also illustrates the distinction between calculation and prediction. Rather than forecasting which securities will outperform, the product focuses on questions that can be answered from information already available about a portfolio. Where the underlying account data does not support a calculation, such as certain cost-basis-dependent measurements, that limitation has to be reflected in what can be reported.
Walnut is one participant in a broader category, and its own comparison of AI investing apps describes how several alternatives approach the problem.
As with any vendor-produced comparison, readers should account for the source when evaluating those conclusions.
A practical standard for AI financial software
The calculation-versus-prediction distinction is useful because it can be applied quickly.
Start by identifying which outputs can be independently verified today and which depend on an uncertain future outcome. Then examine the permissions behind the product, the information it does and does not receive, how errors can be detected and what happens to the data it processes.
Finally, remove the AI interface mentally and examine the underlying product.
For software touching money or sensitive records, those questions usually reveal more than a list of model features.
The most impressive answer is not necessarily the one that sounds the most intelligent. Often it is the one that can show exactly where its answer came from, exactly what it was allowed to do and exactly where those capabilities stop.
This article is informational and not investment advice. Walnut is not a registered investment adviser.
