Prompting, embeddings, RAG, agents, evaluation, safety
Practise these as flashcards. One or two lines each; the ones you miss come back first.
Start freeI would explicitly specify the desired language in the prompt and, if supported by the framework (such as LangChain's llm_kwargs), set the language parameter. Additionally, I could use a system prompt or context examples in the target language to reinforce the expected output.
I would implement a post-processing step that parses the LLM's output and cross-checks any cited chunk IDs against the set of IDs returned by the retriever. If the LLM cites an invalid ID, I would either discard that reference or prompt the model to regenerate the response, possibly using a constrained decoding approach to limit outputs to valid IDs.
Request timeouts often result from cold starts, insufficient worker scaling, or model loading delays. To mitigate these, I would configure autoscaling parameters, use model warm-up routines, and ensure the load balancer's health checks and timeout settings align with the model's initialization time. Monitoring logs for bottlenecks helps identify and resolve specific issues.
A token is a unit of text, such as a word or subword, that the LLM processes. Models break input and output into tokens to manage and generate language.
Temperature controls the randomness of the model's responses. Higher temperature values make outputs more diverse and creative, while lower values make them more focused and deterministic.
Prompt design is crucial because it guides the model to produce the desired output. Well-crafted prompts improve accuracy, consistency, and the ability to generate structured responses.
An embedding is a numerical vector representation of text that captures its semantic meaning. Embeddings are used for tasks like similarity search and document retrieval.
A content filter blocks or flags outputs containing unsafe, offensive, or sensitive information. This helps ensure the application remains compliant and safe for users.
I would explicitly specify the desired language in the prompt and, if supported by the framework (such as LangChain's llm_kwargs), set the language parameter. Additionally, I could use a system prompt or context examples in the target language to reinforce the expected output.
RAG enhances LLM accuracy by retrieving relevant information from external sources and grounding the model's responses in this context. This reduces hallucinations and ensures answers are based on factual data.
Documents are split into chunks, often based on token count or semantic boundaries. Each chunk is embedded, stored in a vector database, and retrieved for queries based on similarity. The most relevant chunks are then fed to the LLM.
Tool calling allows LLMs to interact with external APIs or functions by generating structured requests. This enables the model to perform tasks like fetching data or running calculations beyond text generation.
I use offline metrics like accuracy or F1 score, and sometimes leverage LLM-as-judge to rate outputs. Additionally, I check for hallucinations and compare outputs to ground truth or human judgments.
I optimize cost and latency by batching requests, caching frequent queries, using smaller models when possible, and minimizing the context window to reduce token usage and response time.
Prompt injection can be mitigated by sanitizing user inputs, using strict input validation, and applying output filters. It's also important to monitor for abnormal behavior and restrict sensitive actions.