Anthropic: Many internal products have never been released

Author: Digital Open Source

Recently, Anthropic publicly submitted its S-1 prospectus to the U.S. Securities and Exchange Commission (SEC). The prospectus states that enterprise clients embedding Claude into critical business processes have achieved stronger retention and higher monetization compared to consumer-grade AI. Focusing on the enterprise market and incubating Claude Code are two product practices of Anthropic's research and lab team. Recently, Dianne Na Penn, the product lead of the research and lab team, appeared on the podcast The Skip, where the team is responsible for the core model products from the Claude 2 to Claude 5.5 family and has participated in the incubation and expansion of products such as Claude Code, MCP, Skills, and Claude Design. This conversation explored topics such as the monthly planning and long-term assumptions of research-driven companies, the process of Claude Code evolving from grassroots exploration to product, the trade-offs between research priorities and user feedback, the internal culture of product managers at Anthropic, and the role of front-line deployment engineers (PDE).

Dianne Na Penn stated that Anthropic does not spend three months creating an annual plan that becomes obsolete two weeks later, but instead plans on a monthly basis, based on high-level assumptions about industry trends over the next 6 to 12 months; the product team has also avoided falling into a short-term cycle of "someone released a feature, and it must be done tomorrow."

In product incubation, Claude Code did not suddenly emerge one day, but is the result of long-term trial and error by API clients and internal and external personnel. The interaction paradigm of command-line terminals, combined with allowing the model to execute more Agent loops through the Harness, truly created a chemical reaction. The team had intense discussions about whether to provide fine-tuning APIs, with the core consideration being that model iterations are too rapid, and enterprises often lag two to three generations behind when they attempt to retrain and deploy the model themselves.

Moreover, the trade-offs between research priorities and user feedback rely heavily on ongoing discussions with research leads. The value of product managers lies in clearly articulating the impact of issues on users, rather than dictating training parameters for the research team; the team also copes with the high-intensity work rhythm through collective vacations and pairing two people to be responsible for the same business line.

01 Research-Driven Strategic Rhythm: Monthly Planning with 6 to 12 Months of Assumptions

In the face of a rapidly changing industry, one significant help to the product team is that Anthropic's research organization also possesses mid- to long-term strategic thinking. Dianne Na Penn explained that the company's research mission and North Star goals focus on the long term, creating a beneficial pull that prevents the team from falling into a short-term response cycle of "we released features X, Y, Z today, and we must do the same tomorrow."

Specifically regarding planning methods, the team focuses more on monthly goals, but this is based on high-level judgments about industry trends over the next six to twelve months. Dianne Na Penn said that the team does not spend three months creating an annual plan that becomes obsolete two weeks later. In the early years, they tried a complete OKR process, requiring everyone to align on the same OKRs, but as the assumptions about usage and adoption rates were proven to be severely underestimated, these OKRs were quickly discarded.

The ideal way of working is to first determine core assumptions over a three to six-month dimension, then break them down into monthly goals, advancing towards long-term objectives while maintaining agile responses to the market. For example, in the life sciences field, Dario and others often discuss how AI will play a significant role in helping life science institutions tackle diseases. This is not a short-term project but needs to be translated into concrete actions, such as what model training or product investment tests need to be conducted each month and how to ensure that scientists and researchers can access these models while also responding to early market dynamics.

02 Product Incubation: Grassroots Exploration Driven and What Not to Do

The birth of Claude Code is a sample for understanding Anthropic's product development approach. Dianne Na Penn believes that product development is more driven by grassroots exploration because this technology and its ecosystem have countless possibilities for generalization. Long before Claude Code appeared, API clients were already using the model for code auto-completion; writing code did not suddenly acquire Agent attributes one day, but rather went through different phases of continuous trial and error by internal and external personnel. The command-line terminal interaction paradigm, combined with allowing the model to execute more Agent loops through the product experience's Harness and model capabilities, truly created a chemical reaction.

The judgment of priorities largely depends on the core themes the team insists on betting on and whether they can quickly validate whether the current technology stack is feasible in terms of experience or effectiveness, and whether users feel sufficiently surprised. The current strategy focuses more on B2B and enterprise markets because, compared to the rapidly changing consumer market, the demand on the enterprise side is more stable.

What not to do is equally important. The team had intense discussions about whether to provide fine-tuning APIs, leading to the product thinking that the core model iterates too quickly. By the time enterprise teams figure out how to retrain, optimize, and deploy the model themselves, they may have already fallen two to three generations behind, ultimately leading to a complete restart. Dianne Na Penn thus questioned whether this is a wise investment for the company, users, and clients. Continuously asking "what changes will happen in three or six months" helps the team clarify which products are worth pushing to the market; many internally developed prototypes were ultimately not released externally because the primary goal was to stimulate creativity or conduct technical validation.

03 Research Priorities and User Feedback: How Product Managers Collaborate with Research Teams

Researchers do not only focus on current users. Dianne Na Penn pointed out that researchers in cutting-edge AI labs are not just thinking about existing users in the short term; often, they are looking at yet-to-emerge user groups and the second-order effects of technology, especially in areas like model safety and alignment. When the team internally reaches a consensus that they are serving not only current users but also future user ecosystems, many issues become easier to resolve.

For example, when the product team raises the question of "why can't Claude output JSON or Schema formats better," it may mean that researchers need to pause their work on developing computer control or optimizing reasoning capabilities. This is not something that can improve existing experiences in a step-by-step manner like traditional PM roadmaps; rather, it involves weighing long-term technological bets against current training investments. When evaluating an issue, it is also necessary to assess whether, with the natural evolution of model intelligence, it will automatically downgrade and no longer be a P0 level issue. Dianne Na Penn admitted that this is a frustrating answer for PMs who are used to planning perfect paths.

Taking memory functionality as an example, two years ago, the team often received broad feedback like "Claude does not perform memory functions well." While this feedback is still heard today, the underlying technical complexity is no longer comparable. If poor memory performance directly leads to user attrition, the team will not stand idly by; they may find alternative approaches from a product perspective, such as appropriately limiting usage scenarios, allowing users to still have a decent experience while waiting for the next generation of model capabilities to leap forward. In summary, there are three points: it is essential to consider users at different life cycle stages; it is necessary to acknowledge that this uncertain development can sometimes be frustrating compared to product development based on mature technologies; and most importantly, establish a cognitive model that clarifies ultimate goals and controllable means, rather than blindly believing that modifying underlying research is always the only and optimal solution.

Decision-making primarily relies on discussion. During the training of each new version of Claude, the team holds regular stand-up meetings with research leads to track training progress and discuss what needs adjustment and what is certain. The value of product managers lies in explaining why a particular issue is P0: it has a significant impact on users, and if not resolved, the expected outcomes cannot be delivered; conversely, if complaints are only heard from a small group of users, or if the usage frequency of related features is shrinking, it may be classified as P2, leading to the design of alternative paths. Dianne Na Penn emphasized that PMs do not have the authority to dictate changes to training parameters for the research team, just as engineering teams would not accept being told to write code line by line according to instructions. What PMs need to do is clarify how an issue impacts users and then collaboratively seek solutions; if prompts and other methods do not alleviate the issue, only then does it fall to adjusting model training.

04 Internal Talent Culture and Training System

When hiring, diversity in background is a common point for the team. Many product managers' career trajectories do not follow a linear path within the same company but resemble a cross-disciplinary exploration, which endows them with strong adaptability, as technology is rapidly changing and past assumptions are constantly being broken. Among them, there are individuals who excel at quickly grasping new things or have served as the first product manager in a startup. Dianne Na Penn believes this experience is highly competitive because the first product manager often has to juggle customer success, IT operations, and even business development, and the agility developed from wearing multiple hats aligns well with the team's needs. The team has also hired former researchers and employees who have been at a company for a long time and are eager to break free from inherent perceptions of products and workflows.

After joining, the bar remains high. Being part of the team does not mean that new hires can smoothly return to their old roles on their first day. Dianne Na Penn has made it clear in the new employee onboarding plan that the first two months should be the most challenging period of their careers. If they feel pain, it indicates they are in the right state because the pace is extremely fast, requiring a balance between short-term goals and rapidly changing mid- to long-term plans, while also formulating product plans, strategic roadmaps, and delivering results for the business lines they are responsible for.

The team deliberately arranges highly challenging initial adaptation tasks. Those who can adapt quickly enter a fast track for development, while those who complain "it's too hard" or find various reasons for not being able to keep up often struggle more and more. Within one or two months of joining, it becomes clear who has potential, ultimately testing for a growth mindset.

When asked where those who feel challenged typically get stuck, Dianne Na Penn pointed out that especially product managers from large companies are accustomed to a hierarchical work model: a vice president issues directives, and the execution chain blindly follows, with few stopping to question whether this directive is still the correct choice. This is in stark contrast to academia, where even a first-year PhD student dares to question their advisor.

At Anthropic, employees need to break free from entrenched hierarchical notions and dare to express opinions to Chief Scientist Jared, for example, pointing out overlooked assumptions after reading his article on a certain technology and suggesting adjustments that would be more valuable to users. This requires early participation in discussions rather than assuming that executives have already thought everything through; expression must be based on curiosity and mutual trust, rather than arbitrarily accusing others of being wrong without providing detailed evidence. What the team seeks are individuals who can engage in deep discussions, provide new perspectives, iterate on existing foundations, and drive implementation. The ideas initially thrown out may be those considered two or three years ago but set aside, but if someone is willing to put in the effort to research, question established assumptions in a respectful manner, and offer unique insights, they often achieve great success.

The root of this atmosphere, Dianne Na Penn believes, is related to the frequent iteration of models, but ultimately stems from the corporate culture: the company started as a research lab, and many co-founders highly value first principles. The breakthroughs in technology today are precisely because they have not been limited by past assumptions, and the courage to question assumptions and validate them through hands-on experimentation is the core operational logic among top researchers. Internal communication is extremely transparent, with most channels completely open. When discussing a product detail in a group, someone you may not have noticed might politely interject, asking if everyone has considered an alternative solution. The team values collaboration that affirms each other and continuously expands ideas.

About successful product managers, there is one more point: an almost obsessive enthusiasm for the technology. From the moment they join, they deeply test their own product features during the prototyping phase or repeatedly test the latest models and publicly share their evaluation insights, proving how to optimize products and models through action rather than just talk. Compared to hiding in a corner writing requirement documents, actively pushing for the implementation of models and products is a hallmark of top product managers.

Many junior product managers complain about the poor environment and the scarcity of good positions, but Dianne Na Penn's observations are quite the opposite: product managers with less experience in the team demonstrate extremely high work efficiency and limitless potential because they do not need to deliberately break free from the constraints of the established organizational structure. Here, they do not have to wait for others to issue orders or for the boss to sign off on requirement documents to move forward. Those who are highly motivated and proactive think, "This is my module, and I must ensure its implementation," actively communicating with clients and connecting with researchers, regardless of whether the other party is the research supervisor.

Compared to product managers from large company backgrounds, these individuals do not need to go through as many processes of aligning various teams and following step-by-step verification procedures, which often makes them highly efficient. However, this does not mean that mid to senior-level product managers lack these qualities; the team includes both former senior product managers who have started their own businesses and those who have tackled challenging projects internally and fought their way through.

In Dianne Na Penn's view, the core lies in whether one has accomplished something that requires overcoming significant organizational inertia and resistance. That kind of perseverance and mission-driven attitude is often more lethal than any other trait, which means maintaining curiosity, seeking answers by all means, and achieving self-drive when facing unknown challenges, rather than passively waiting.

In terms of training, the team's starting point is the definition of product management. Dianne Na Penn states that it is essential to return to the initial state of a new technology rather than mechanically defining job responsibilities. The core principle still revolves around the user, specifically how to create value for users in a way that has a compounding effect, with all other actions deriving from this. Perhaps a year from now, writing model evaluations will no longer be the best way to define user outcomes, or perhaps everyone will need to write code to build prototypes directly, but the North Star goal remains unchanged.

The onboarding process is extremely high-pressure: new employees must complete user research, write model evaluations, and develop product roadmaps within the first two to three weeks. Many often ask if they can have an additional two weeks, but this is precisely to force out execution power, pushing them into a practical state where they must keep pace with the company's rhythm, which has proven to be very effective. At the same time, newcomers are placed directly onto the highest priority core business line, and the team tends to remain small, allowing each person to simultaneously handle two or three directions, which forces cross-departmental communication, idea collisions, and rapid learning, also creating a strong sense of delivery urgency.

The team also holds a training camp once a year, focusing on sharing how to create products capable of driving exponential growth, meaning that as the underlying capabilities of the model improve, the product value will also explode exponentially. The content includes how to optimize Claude, write high-quality prompts, and conduct model evaluations. After recording the courses, they become mandatory for every new employee. This initiative began in 2024 when the team had only four or five people, and daily work had started to resemble that of research-oriented PMs, such as reading user-authorized feedback data and writing assessments. Since it greatly benefits product functionality, they wanted more people to master it, so they documented the operational steps, key metrics, and less valuable parts. Initially, it was only aimed at PMs outside the research-oriented product team, but it is now open to all employees at Anthropic.

The training camp is also an opportunity to build an open Q&A platform. Dianne Na Penn emphasizes that PMs do not have to simply wait for model iterations; they can achieve desired results by modifying prompts. Live demonstrations lowered the threshold and broke the mystique of technology, with even the sales team participating and exclaiming, "So this is evaluation, this is a prompt, I can do this too." These methods can be completely replicated and scaled, and any company can do it.

Regarding the mentorship system, new employees are paired one-on-one with senior team members and assisted in onboarding using internal tools like Claude Tag. Newcomers are often advised to review the group chat records from the past two months because joining the company is like entering an ongoing lively conversation. The most efficient approach is to do thorough homework first, express their understanding of the business background, and seek opinions, actively integrating rather than waiting for the company to feed knowledge to them.

05 Frontline Deployment Engineers: Trade-offs Between Depth and Breadth

Regarding whether product managers should dive deep into the front lines and embed themselves in customer scenarios, Anthropic has equipped frontline deployment engineers and a dedicated frontline deployment team to assist enterprise clients in large-scale implementation on the platform or application side. Dianne Na Penn agrees that this skill of deeply engaging with the front lines is essential, but the reality is that if one digs too deep on one end, they inevitably have to compromise on the other. This often leads to a feeling of being torn: wanting to keep up with every iteration of the research model while also wanting to listen to every customer's feedback, but currently, it is impossible to sit in on every customer conference call.

The key issue is how to effectively feedback the deep insights from the business front lines and feed them back to the products they are responsible for. The team collects feedback directly from API clients and early adopters of new model capabilities, but energy and capacity do not allow for a fully dedicated frontline deployment, so they closely collaborate with the frontline business team, using their insights as supplementary validation. The current model focuses more on panoramic sharing of business context rather than requiring every PM to be deeply rooted in customer front lines while also aligning with top research teams, which makes achieving extreme depth of expertise on both ends extremely difficult.

An effective experience is that the team later selected a group of individuals from the internal application and frontline deployment engineering teams to transition to PM roles, and their performance was impressive: they have a deep business sense, a thorough understanding of the principles of Harness, the application architecture on the user side, and the motivations for use, enabling them to smoothly relay frontline voices back to the R&D team. Dianne Na Penn recalls that when Claude.ai was first launched in 2023 and was still the only internal testing product, she personally wrote the system prompts for a long time before handing them over to the newly joined PMs. This is essentially the daily routine of a frontline deployment engineer. Discussing technical depth abstractly often feels empty; to make it concrete, perhaps the most solid approach is to become a frontline deployment engineer and closely listen to customers, which might be an answer to reshaping the PM career path.