Unlock Gemini’s Mind: See Why Your AI Makes Its Choices with Logprobs — Part 1
If you have ever conceived of AI as being a block box you are working with …. Then check out this new feature we have released in Vertex AI for Logprobs.
In your role as a developer, you build applications powered by large language models, but when Gemini gives you an answer, have you ever wished you could ask, “Why did you choose that [token] word as the next output [token] word?”
For many, the model’s decision-making process felt opaque. You received the output, but not the reasoning behind it. Here is some additional visibility you can have today.
We’re excited to pull back the curtain with the official introduction of the logprobs feature in the Gemini API on Vertex AI. This isn’t just a debugging tool; it’s a window into the model’s reasoning, allowing you to build smarter, more reliable, and more context-aware applications than ever before.
What Exactly Are Logprobs? Demystifying the Score
Before we dive into the code, let’s break down what a “logprob” actually is.
At its core, a logprob is the natural logarithm of the probability score the model assigned to a specific word (or “token”).
Here’s the simple version:
- Every time Gemini generates a word, it first calculates the probability for all possible next words.
- A probability is always a number between 0 (impossible) and 1 (certain).
- The natural logarithm of a number between 0 and 1 is always negative.
This leads to one simple, powerful rule: The closer a logprob score is to 0, the more confident the model is in its choice. A score of -0.01 is extremely confident, while a score of -10.0 shows very low confidence.
Putting It Into Practice: Enabling Logprobs
Getting started is easy. You can enable log probabilities by setting two simple parameters in the generation_config of your API request.
- response_logprobs=True: This tells Gemini, “Yes, please show me the scores for the words you chose.”
- logprobs=[integer]: This is where it gets really interesting. You can ask the model, “Also, show me the scores for the top 3 (or 5, or 10) words you didn’t choose.”
Let’s see it in action. Imagine we want to classify a sentence as “Positive,” “Negative,” or “Neutral.”
from google.genai.types import GenerateContentConfig
# Define our prompt and the only possible answers (the schema)
prompt = “I am not sure if I really like this product a lot.”
response_schema = {“type”: “STRING”, “enum”: [“Positive”, “Negative”, “Neutral”]}# Make the API call with logprobs enabled
response = client.models.generate_content(
model=”gemini-2.5-flash”,
contents=prompt,
generation_config=GenerateContentConfig(
response_mime_type=”application/json”,
response_schema=response_schema,
response_logprobs=True, # Turn on logprobs for the chosen token
logprobs=3, # Also show the top 3 alternatives
),
)
When we process this response, we might see something like this:
- Chosen Token: ‘Neutral’ (-0.0214)
- Alternative Tokens:
- ‘Positive’: (-4.8219)
- ‘Negative’: (-5.6293)
Instantly, we see that Gemini was overwhelmingly confident in its choice of “Neutral,” as its score is incredibly close to 0. The other options weren’t even close.
The Future is More Transparent
Logprobs are more than just numbers; they are a fundamental shift toward building more transparent, reliable, and intelligent AI. By understanding the why behind a model’s choices, you can debug more effectively, build user experiences that feel like magic, and create systems you can truly trust.
These are just a few examples to get you started. What will you build with this newfound insight?
To dive deeper, explore the full Introductory Notebook and consult the official Gemini API documentation.
