Ambient "operator recommender" — predictive next-operator suggestions on the canvas

[gupta-sahil01]

The idea in one line

As a user builds a workflow, show 1–3 faded "ghost" operators on the output port of the operator they just added — ranked suggestions for what likely comes next, one click to insert.

How it differs from the existing chatbot agent

The current chatbot is on-demand and conversational: the user describes a goal, the agent generates a workflow. That's powerful, but it assumes the user already knows what to ask for. This feature is ambient and predictive: suggestions surface automatically as part of normal canvas building, with no prompt. It's aimed at the moment where a user has some data on the canvas but doesn't know which operator unlocks the next step. Think Copilot ghost-text, but for the dataflow graph rather than a chat box. The two are complementary — I'd want to reuse the chatbot's LLM plumbing rather than build a parallel stack.

Questions for the maintainers

Existing work: Is anyone already working on operator suggestions / autocomplete on the canvas? I'd rather extend than collide.
Mergeability: If this turns out well, would the team be open to a PR? What would make it acceptable (config flags, opt-in, evaluation, etc.)?

Quick visuals:

The left panel is Texera as it exists today. The user either drags operators by hand from a side panel, or types a description into the chatbot, which constructs a workflow and offers to apply it. Both modes assume the user already knows what to ask for.

The right panel is my idea. The user has only built two operators (CSV Reader → Filter). The pulsing teal port on Filter's output is where your feature lives — three faded "ghost" operators appear automatically, ranked by how well they fit the data. Hovering shows the rationale. Clicking a ghost solidifies it into a real operator on the canvas. No chat needed.

[chenlica]

This is a great idea! It will definitely improve the user experience. Agreed that it complements the chatbot, so there is no conflict. Here are a few comments:

• The only related effort I can think of was a research project by @seongjinyoon with this autocomplete feature as a motivation. It didn't have a PR. @seongjinyoon can chime in and confirm.

• I think the UI is not hard. A key question is the backend. What's your thought on how to recommend operators? Using an LLM? Or developing a new recommender?

[gupta-sahil01]

[chenlica]

@gupta-sahil01 Thanks for the diagram. A general principle we want to follow is to make each feature easy to develop and maintain. Please provide more details, such as: 1) any offline analysis needed; and 2) what information should be stored and where.

[gupta-sahil01]

1. Offline analysis
   Three small text files we write once and update as the operator catalogue changes — no model training:

a short description of each operator (so the LLM knows what each one does)
a handful of example patterns ("after a Filter on text, suggest Sentiment") to guide ranking
a small test set of workflows to check ranking quality in CI

Adding a new operator just means adding one line to the descriptions file. CI catches it if you forget.
2) What's stored and where
Two places:

In the repo (versioned with code, reviewed in PRs): the three files above, plus the LLM prompt templates. That's the entire "knowledge base."
In memory at runtime (auto-evicting cache): recent suggestion results, and one-time analyses of any UDF source code we've seen. Caches a recent UDF summary so we don't pay the LLM to re-read the same Python twice.

No database, no model registry, no separate service. Four files in the repo + a cache.

[zuozhiw]

I like a stateless backend api just takes in the current state and returns recommendations. Stateful things are harder to get right.

Back on to how this can be implemented, I'm mostly leaning towards LLM based recommendation. I'm not a huge fan of training a custom model for recommendations, if we use LLMs, we get a free ride as model capabilities increase.

My main concern is that if the speed of LLMs are fast enough, also if we bring LLMs to the table, we might need to maintain the conversation history somewhere for it to be more cache friendly.

I would really recommend looking into how cursor does code autocomplete. I know that cursor trains its own small mod (I think it's a small LLM model) and it's both fast and accurate. If we can get some more context on how cursor's autocomplete work today I'll be more comfortable. Also I would really like to see if there are any online articles discussing using LLMs to run code autocomplete.

[gupta-sahil01]

Thanks for your valuable advice. I'll surely look into the auto-completion of code in cursor. That looks like something very similar to what I've proposed here

[chenlica]

@gupta-sahil01 Do you have any update?

[gupta-sahil01]

I'm still looking into the auto code completion of cursor. Right now, from what I've understood, is that cursor uses a small custom-trained model (not GPT or Claude) fine-tuned specifically for code completion, with a ~272k-token context window over your current file, open tabs, and recent edits.

There's a neat trick called "speculative edits", which means that they feed your existing code back to the model as draft tokens, so it only emits the bits that change, about a 13× speedup, which is what lets full-file rewrites feel instant.

[zuozhiw]

Hi @gupta-sahil01 thanks for looking into cursor, I think a custom-trained model is not very feasible for us, so for this one we should still stick to openly avaiable models. But cursor uses a smaller model for this task, so we should also consider using less powerful models, which are both cheaper and possibly faster. With this in mind I would suggest we proceed to next steps and do not stuck in researching for too long. Thanks. What would be a sensible next step?

[gupta-sahil01]

Sensible next step: I'll open a small end-to-end PR that wires up a stateless /recommend endpoint in the existing agent-service (reusing the compile API for schemas, the operator catalog, and the same AI SDK + LiteLLM gateway the chatbot already uses), and a frontend hook that listens for "operator added" and feeds suggestions into the existing ghost-operator UI from drag-drop. First version returns hardcoded suggestions so we can confirm the wiring works; second version swaps in a real LLM call with a small/cheap model.

Would that be ok?

[zuozhiw]

@gupta-sahil01 the plan sounds good, please go ahead with first steps so that we'll be able to see the API shape, thanks!

[gupta-sahil01]

I have a question. How do I start building? Do I just show the UI part with the API calling, because for that I'll need credits, right? I have finished setting up all the microservices and frontend, and it's running successfully on my laptop. Should I set up the texera agent as well?

[chenlica]

@xuang7 @yangzhang75 @kz930 and others: Please chime in.

[gupta-sahil01]

Hey, any help would be appreciated.

[xuang7]

The idea sounds very good! This would significantly improve the usability of workflow building, and the plan discussed above seems like a solid way to get there incrementally.

As discussed, version 1 can return hardcoded recommendations, so the /recommend endpoint does not need to call an LLM yet. This allows you to wire up and demo the full loop with zero API cost: the frontend listens for the "operator added" event, calls the endpoint, and renders the ghost suggestions. The agent-service should still run locally, since that is where the new endpoint should live, but for version 1, the default configuration should be sufficient because no LLM call is involved.

For version 2, which connects to a real LLM, you could follow the suggestions earlier in this thread: reuse the same gateway that the chatbot already uses, and start with a smaller, cheaper model. That part can come later after we get more input from the community.

[gupta-sahil01]

Hey, appreciate your thoughts. I've opened an issue to track this - #6293