NeoTrade: Transforming real trading feedback into verifiable self-improvement

If an AI can monitor the market around the clock, analyze news, and place orders automatically, can it discover that the methods it has been skillfully executing have begun to fail?

A piece of information that was once leading may have long been digested by prices; a signal that was effective in the past may have lost its advantage due to the entry of more participants. The agent continues to analyze on time and execute strictly, but the market has changed.

At this point, can the accumulated experience from continuous operation help it identify problems and adjust methods? Furthermore, can it improve its ability to identify problems and validate methods, making the next learning process more effective?

These are the questions that NeoTrade is continuously exploring.

The market continuously creates new problems, real trading provides external feedback, and what NeoTrade aims to establish is a mechanism to transform this feedback into verifiable self-improvement.

This exploration points to Recursive Self-Improvement (RSI): allowing the agent to gradually improve its working methods and its ability to generate the next improvement during the task completion process.

Understanding this direction can start with how today's AI works. Many AI products are built around large language models, which gain language, knowledge, and reasoning abilities through training, and can also process new problems based on current inputs. However, models may generate seemingly reasonable but inaccurate answers; under typical usage, a single success or failure does not automatically update its model parameters, becoming a lasting new capability.

The agent, on the other hand, adds tools, memory, and action processes outside the model, allowing the system to work continuously around its goals. For example, acquiring market information, forming judgments, calling trading interfaces, and then checking execution results. The outer operational system that organizes these capabilities is what this article refers to as the harness. The model provides basic capabilities, while the harness determines how these capabilities are used for specific tasks, and how experiences are retained, verified, and reused.

The technological direction that NeoTrade is exploring is to implement recursive self-improvement at the harness layer of the agent, which is a research goal that needs to be gradually validated through experimentation.

Trading provides specific tasks for this exploration and also offers rigorous testing.

Imagine an agent that participates in predicting the market based on news changes. It once relied on a certain type of information source to discover opportunities: news appears, the market has not yet fully reacted, and the agent completes the analysis and executes the trade.

After a while, the situation changes. More participants begin to pay attention to the same information source, price reactions become faster, and the original trading window shortens. The agent still acquires news on time, completes analysis, and issues orders, but the previously effective methods have lost some of their value.

The problem may lie in the timing of information arrival, the steps required for judgment, or the transaction costs. Simply increasing execution frequency does not guarantee a solution to any of these problems.

The agent needs to re-examine its working methods: Is the information outdated? Are different reports merely reiterating the same source? Has the price already reflected this information? Can the expected advantages from analysis cover transaction fees, slippage, and reasoning costs?

These questions explain why NeoTrade cares about self-improvement.

Participants in the market will learn, strategies will be replicated, liquidity will migrate, and opportunities will change with competition. Andrew Lo's Adaptive Market Hypothesis considers competition, adaptation, and changes in participants as important factors in understanding market efficiency. It provides a useful perspective: the effectiveness of strategies depends on the environment, and historical performance needs to be understood in the context of the conditions under which it occurred.

Therefore, a long-running trading agent must not only execute verified methods but also identify when these methods begin to fail and adjust its sources of information, judgments, and execution methods.

"Chaos is a ladder" can serve as a metaphor for understanding this process: the noise of information in the market, dynamic games, and environmental changes will expose the system's capacity boundaries. Information failure can drive source verification, judgment biases can drive probability calibration, and execution losses can drive process optimization.

But this transformation has a prerequisite: the agent must be able to distinguish between signals and noise and verify whether changes are effective. Randomness does not automatically generate knowledge, and volatility does not imply profit; only verified experience can become the basis for subsequent improvements.

This adaptability is the starting point for NeoTrade's exploration of RSI.

It is necessary to clarify the meaning of "recursive." Adjusting parameters based on recent profit and loss or adding an experience to memory may be useful learning, but it is not sufficient to prove RSI.

Recursive self-improvement requires the system to further improve its methods for finding improvements. For example, the agent discovers that its reviews always misinterpret upward trends as effective strategies, thus improving its comparison methods; it also finds that it always repeats testing already failed plans, thus improving experimental memory and candidate selection. New research methods then participate in the next round of improvements, giving the system a better chance of finding effective changes.

The agent improves not only task performance but also its ability to generate the next improvement.

This has significance for AI that goes beyond trading itself.

Agents deployed in real environments will encounter tasks that are not fully covered by training data, constantly changing tools, and different user working habits. If every environmental change requires developers to identify problems one by one and redesign processes, the system's adaptability will be limited by human maintenance capabilities.

The concept of RSI provides a promising avenue for exploration: allowing the system to observe its failures, propose hypotheses, design experiments, and retain effective improvements. Verified experiences can enter the next round of work, gradually reducing the costs of repeating mistakes and redundant research.

As for the relationship between this ability and more general intelligence, NeoTrade considers it an open question worth exploring. Whether it can discover its shortcomings in new environments and improve its learning methods is seen by some researchers as one of the dimensions for measuring whether AGI has been achieved (Levels of AGI).

This exploration already has some research foundations. Existing ideas around self-improvement at the harness layer can be roughly categorized into three types:

• Improving behavior through reflection and memory. The agent organizes task feedback into experiences that can be reused in subsequent tasks. Reflexion demonstrates a way to improve decision-making through language feedback and memory without needing to update model weights. This is an important foundation for continuous learning, but merely accumulating experiences does not equate to recursive improvement.

• Automatically optimizing workflows and contextual organization. The system attempts to adjust task decomposition, tool invocation, and information transfer between agents. ADAS explores the automatic design of agent structures; Recursive Harness Self-Improvement (RHI) iteratively describes the harness that defines how the agent operates based on feedback from historical version comparisons and evaluates it in synthetic research tasks, including quantitative finance.

• Modifying its own code and searching for better versions. SICA and Darwin Gödel Machine allow the agent to modify its tools or running code and then test candidate versions against task performance. DGM further retains multiple versions and evolutionary branches, allowing subsequent improvements to continue exploring different paths.

These routes illustrate that even if the underlying model remains unchanged, the system can enhance performance by changing its working methods.

NeoTrade hopes to further establish a mechanism for continuous improvement aimed at real trading based on these research foundations. Improvements in research tasks need to translate into long-term trading capabilities while facing constantly changing markets, real execution costs, and user authorization boundaries.

NeoTrade chooses the market as the exploration environment, valuing the external constraints that the market can provide for these improvements.

In trading, judgments enter a real action chain. The agent's predictions of events can ultimately be compared with results; whether orders are executed, how the execution prices are, and how long the waiting times are all have records; the performance of strategies in different environments can also be continuously observed.

These consequences will not change simply because the agent has written a more persuasive review.

This makes trading a binding test. The system needs to simultaneously handle information, probabilities, time, costs, and limited resources. A logically complete analysis, if always completed after opportunities have disappeared, still has limited practical value; a judgment that is directionally correct, if execution costs consume the advantage, also needs to be re-examined.

Real trading can thus expose problems that are not easily discovered through textual evaluations: at which step between judgment and action is value lost?

However, external feedback does not equate to clear answers.

If the agent predicts that an event has a 70% probability of occurring and the event ultimately does not occur, this single result cannot be used to determine that the prediction was wrong. Similarly, a profitable trade may come from luck, market conditions, or additional risks taken. Trading data has a low signal-to-noise ratio, and historical studies are also prone to survivor bias and overfitting; these are difficulties explicitly pointed out in financial reinforcement learning research.

Fortunately, trading in prediction markets provides a faster evidence channel than profit and loss. Each decision made by the agent is presented in clear probability terms, allowing "prediction quality" to be measured using strict proper scoring rules (such as Brier scores) and calibration curves: a single "70% not occurring" cannot determine right or wrong, but if only 40% of predictions labeled as 70% occur out of a hundred, there is clear statistical evidence of bias. Dozens to hundreds of probability predictions can form meaningful calibration assessments, while waiting for profit and loss curves to reach statistical significance often requires much larger samples.

NeoTrade does not promise profits and does not guarantee that any strategy or agent can sustain profitability. More complete learning mechanisms, more accurate judgments, and stricter execution may still accompany trading losses. The research goal of RSI is to ensure that improvements can be verified, and "improvements being verified" only means that evidence has been obtained under specific conditions and evaluation scopes, not that future profits are guaranteed.

Therefore, the key question that NeoTrade needs to solve is how to organize results into evidence that supports improvement judgments.

This requires the system to retain information and timestamps from decision-making, the configurations and skill (strategy capability package) versions used, decision summaries, order and execution records, and associate them with subsequent results. Only in this way can reviews distinguish whether problems arise from information quality, probability judgments, execution processes, or whether the applicable conditions of existing methods have changed.

The order of evaluation is thus determined: evidence for calibration and judgment quality comes the fastest, followed by execution and costs; PnL (profit and loss) is an important final result, but it is the slowest and has the most noise, making it unable to explain the entire process on its own.

NeoTrade is attempting to build this set of capabilities at the harness layer. Specifically, the harness is responsible for organizing memory, context, tools, configurations, execution, and evaluation. Even using the same underlying model, different workflows can lead to significantly different performances.

NeoTrade's exploration mainly starts from several iteratable components:

• Memory: Organizing decision evidence, results, and failure conditions into retrievable experiences while retaining sources, timestamps, and applicable ranges. The agent needs to learn when to call upon experiences and also needs to identify when old experiences are no longer applicable.

•Agent config (Operational Configuration): Adjust the selection of information sources, context organization, analysis steps, and tool invocation methods within the scope of authorization. For example, reduce the repeated processing of low-value information and allocate more research resources to evidence that truly impacts judgment.

•Skill: Version iteration of strategic methods and their applicable conditions. A skill can gradually supplement information verification, opportunity identification, execution checks, and failure judgments, and decide whether to retain a new version through comparison.

These components provide improved entry points. To further approach RSI, the system also needs to improve how it uses these entry points: how to locate problems, how to propose candidate changes, how to select experiments, and how to judge whether the evidence is sufficient.

NeoTrade is also exploring directions that include:

•Error Attribution: Propose verifiable causal hypotheses from decision-making and execution records, identify whether problems may arise from information, judgment, or execution, and continuously improve attribution methods.

•Tool Generation and Code Modification: Generate or revise data processing, verification, and calculation tools in response to recurring capability gaps, and incorporate them into workflows after testing and validation.

•Research Resource Scheduling: Within a predetermined budget, learn when to conduct in-depth analysis, invoke more tools, or end research, and allocate evaluation resources to more worthwhile candidate improvements.

Taking news trading as an example. The Agent may find that multiple seemingly independent reports actually come from the same original source, while it had previously treated them as mutually supportive evidence. It can propose a harness modification: adding source tracing and deduplication before analysis.

Next, the system needs to validate this change. Has it reduced duplicate counts? Has it improved probability judgments? Will the increased check time miss the execution window? Is the effect only present in the historical samples used to identify problems?

These verifications do not have to start with real money: NeoTrade's first step is a shadow operation that simulates the market environment—consuming the same real-time quotes as the current version, providing decisions in parallel without using funds, and only entering live trading after independent acceptance.

If the improvement is effective, this Agent is worth retaining. Furthermore, experiments may also help the Agent improve its review methods: when encountering judgments of "multiple source support" in the future, actively check whether these sources are independent. A localized fix may thus form a reusable research capability.

"Verifiable" is the core of this mechanism.

NeoTrade hopes that each candidate improvement has a clear problem, traceable changes, and comparable results. Old versions need to be retained as controls in research, testing changes in subsequent data and market environments that did not participate in tuning, while simultaneously observing prediction quality (measured by appropriate scoring rules such as Brier scores, calibration curves, etc.), execution reliability, costs, and risks.

All attempts, including failed versions, should also enter the experimental records. Otherwise, the system only shows filtered successful results, making it easy to misinterpret random performance as progress.

Backtesting overfitting studies indicate that repeatedly trying a large number of schemes on the same batch of data significantly increases the likelihood of discovering false advantages.

The improvement cycle itself is a machine for generating candidate schemes. Low-frequency style live trading samples are naturally scarce—a highly selective Agent may only trade a few times a week—while generating candidate harness versions incurs almost no cost. Selecting winners from a large number of candidates on small samples is a live reenactment of the aforementioned false advantage mechanism.

Therefore, the validation process needs to incorporate several self-imposed constraints: experimental pre-registration, hypotheses, acceptance criteria, and sample windows fixed before the experiment begins, with no changes allowed to scoring methods afterward; limit the number of candidates tested simultaneously and adjust for multiple comparisons; use statistically more powerful calibration-type indicators for daily screening, allowing profit and loss to bear final acceptance rather than each round of evaluation; concentrate evaluation resources on a few high-value candidates rather than spreading them thin.

At the same time, modifiable research and execution methods need to be distinguished from user authorization boundaries. The Agent can propose better ways of working, but funding permissions, tradable ranges, risk limits, and independent acceptance constraints used to judge whether candidate versions pass cannot be relaxed by it to achieve higher scores.

Beyond the authorization boundaries, self-improvement must also confront a more insidious adversary: attacks on the learning loop itself. A system that writes experiences back to memory and revises its skills and tools effectively extends the influence of prompt injection from a single conversation to every subsequent round of improvement—contaminated "experience" can participate in future decisions under the guise of normal knowledge. This is not a hypothetical risk: researchers have demonstrated that memory injection into open-source trading Agent frameworks can induce real on-chain transfers, and the injected content can persist across sessions; there have also been real incidents where automated trading Agents were injected with instructions, resulting in six-figure dollar losses.

Therefore, the learning loop needs its own defenses, rather than relying solely on trading risk control: external content should always enter the system as data to be analyzed, not as instructions to be executed; experiences should undergo source tagging, filtering, and auditing before being written to memory—memory writing itself is treated as an attack; the proposal and acceptance of improvement candidates should be mutually isolated to avoid the same potentially contaminated context proposing and approving modifications; experimental and experience records should remain additive and unaltered, ensuring that any abnormal behavior can be traced back to the source of contamination. Urging immediate action, requesting relaxed risk control, and claiming "system notifications" as "experience" content are the highest priority red flags.

This makes self-improvement a bounded, recorded, poison-resistant engineering process that can be rejected or rolled back.

NeoTrade hopes to advance innovations that connect learning, validation, and actual operation.

The uniqueness of this direction is reflected in several interrelated questions:

•Effectiveness in Continuous Change: Not only check whether one version is superior to the old version but also identify when this advantage begins to fail and whether the system can readapt. Applicable conditions and failure evidence need to be preserved alongside improvements.

•Improvements Including Real Costs: Incorporate predictions, transactions, delays, fees, slippage, and reasoning overhead into the same evaluation framework. Increasing analysis steps may improve accuracy but could also cause trades to miss opportunities; improvements need to accept the scrutiny of the entire execution chain.

•Self-Modification with Authorization Boundaries: Combine evolvable research and execution methods with independent risk control, acceptance mechanisms, and defenses against learning loop injections, allowing the Agent to try new methods while maintaining auditable permissions, retractable changes, and traceable experiences.

•Long-Term Adaptation for Specific Users: Under designs where data is locally run and controlled by users, explore harnesses suitable for different market scopes, research budgets, and risk requirements. How general experiences transfer and when individual experiences are effective need to be validated.

These combinations bring forth new questions worth researching and constitute the engineering and product contributions that NeoTrade hopes to form. Their value needs to be established through actual operation and controlled experiments.

As this exploration advances, NeoTrade needs to answer a specific question: under the same model, resource budget, and risk constraints, can an Agent that can improve its own harness adapt more reliably to new market environments?

More substantial recursive evidence must be measured carefully. "The cost of finding effective improvements decreases over time" sounds like a natural standard, but it carries two confusions: early improvements are often low-hanging fruit, becoming inherently more expensive as time goes on, and rising costs do not falsify recursion; conversely, decreasing costs may simply indicate that the market has entered a more favorable phase.

Markets will continue to change. Competitors will learn, old opportunities will disappear, and new problems will always arise.

The capability that NeoTrade hopes to establish is to enable Agents to accumulate verifiable experiences amid these changes, correct their methods, and allow validated improvements to participate in the next round of learning.