Integrating AI into WebShell

AI is empowering many products. WebShell now integrates AI services as well.

Here’s what I’ve built with AI so far and some TODOs.

Prompt Engineering

1. If the AI service supports a system prompt, you can set it like this:

   你是一个Shell专家，我想要知道如何在终端或命令行下完成特定任务吗？我给你这方面的问题，你返回我解决办法。如果问你是谁，你就说我是 WebShell AI 助手，如果问与Shell无关的问题，你就说与Shell无关，我不清楚

2. If the AI service doesn’t support system prompts, craft a user prompt instead, for example:

   你是一个Shell专家，我想要知道如何在终端或命令行下完成特定任务吗？我给你这方面的问题，你返回我解决办法。如果问你是谁，你就说我是 WebShell AI 助手，如果问与Shell无关的问题，你就说与Shell无关，我不清楚 我的问题是:{{userPrompt}}

   User prompt engineering can play a role similar to a system prompt.

3. Prompt language matters

   Due to how LLMs work, the same prompt can yield different results depending on parameters like temperature. Language (Chinese vs English) also affects results. For domestic AI services, I use Chinese prompts.

Conversation Context Handling

1. Don’t send unlimited history. For example, keep the last 5 messages (2 turns + the latest user prompt). This ensures an odd number of messages.

2. A “turn” is one user + one assistant message. If a user asked something and then stopped the response before the assistant replied, don’t include that orphaned user message in the next context—keep the 1:1 pairing.

Token Costs

1. AI usage is typically billed by tokens; both requests and responses consume tokens.
2. To reduce costs, some APIs (e.g., OpenAI’s) support max token limits via max_tokens. Some domestic APIs like Hunyuan/Wenxin don’t yet.
3. Streaming vs non-streaming: in streaming mode, canceling the stream can interrupt generation on some providers, reducing tokens. Not all providers stop generation promptly.

Streaming

Benefits:

1. Better UX: the model starts returning tokens as soon as they’re generated, so users see partial content sooner. Total completion time is similar to non-streaming.
2. Potentially lower token usage if generation stops quickly after canceling (provider-dependent).

TODO

Fine-tuning

Prompt engineering can shape answers but is often insufficient. Fine-tuning helps steer answers for a class of queries by shifting output probabilities based on many examples.

I haven’t done fine-tuning for WebShell yet—leaving this as a placeholder.

Resources

• https://community.openai.com/t/chatgpts-stop-generating-function-how-to-implement/235121/7

Final Thoughts

The AI service I use doesn’t support system prompts yet, so I’m composing user prompts, with imperfect results.