What if we told you that a model can read a fact and still miss it? In 2023, researchers found that language models answered best when the key fact sat at the start or the end of a long prompt, and worst when it sat in the middle. So we hid five facts from Amazon's 10-K in a long stack of real filing pages and moved each one from the first page to the last. But here's the real mystery: would a 2026 model still lose the middle?
Long context means a prompt with many thousands of tokens, like a whole section of a 10-K. In simple words, it is a lot of reading before the model writes anything. In this blog, we will test where Qwen 3.8 finds a fact in a long prompt, and whether the question should go before or after the document.
The fact page moves. Everything else stays the same.
Setup
We use Qwen 3.8 27B in Ollama, with thinking off and the same answer on every run. We set num_ctx to 131072, so a very long prompt fits, and num_predict to 60, since each answer is one short sentence. We read the 10-K PDF with pypdf. Let's see the code as below:
import re
import ollama
from pypdf import PdfReader
MODEL = "qwen3.8:27b"
def ask(prompt):
response = ollama.chat(
model=MODEL,
messages=[{"role": "user", "content": prompt}],
think=False,
options={"temperature": 0, "seed": 7, "num_ctx": 131072, "num_predict": 60},
)
return response.message.content, response.prompt_eval_countWhat Is the Lost in the Middle Problem?
In 2023, Nelson F. Liu and colleagues published "Lost in the Middle: How Language Models Use Long Contexts" (arXiv 2307.03172), later in the journal TACL. They gave models many documents with one answer hidden among them, and moved that answer from the first document to the last. Accuracy formed a U shape: high at the start, high at the end, and lowest in the middle.
For finance work, this matters a lot. A RAG app or an analyst assistant often pastes many 10-K pages into one prompt. If the model skims the middle, a fact on page 15 of 30 may be missed even though it is right there.
Models have changed a lot since 2023. So let's measure it ourselves.
How Do We Pick Facts the Model Cannot Already Know?
A position test only works if the model needs the document. If it already knows the answer from training, it can answer from memory wherever the page sits. So we pick five small facts from Amazon's 2024 10-K and first ask about each one with no document at all.
pages = [" ".join(page.extract_text().split()) for page in PdfReader("data/amazon 10-k 2024.pdf").pages]
# question, answer pattern, a phrase that marks the page holding the fact
facts = [
("How many shares of Amazon common stock were outstanding as of January 29, 2025?",
r"10,597,729,352", "10,597,729,352"),
("By how much does Amazon expect shortening server useful lives from six to five years to reduce 2025 operating income?",
r"0\.7 billion|700 million", "useful life will change from six years to five years"),
("By how much did changes in foreign exchange rates impact AWS operating income in 2024?",
r"240 million", "positively impacted operating income by $240 million"),
("By how much did changes in foreign exchange rates reduce Amazon's sales and marketing costs in 2024?",
r"263 million", "reduced sales and marketing costs by $263 million"),
("What was the 2024 valuation loss on Amazon's equity investment in Rivian?",
r"1\.6 billion", "$(1.6) billion from our equity investment in Rivian"),
]
home = {q: [i for i, page in enumerate(pages) if anchor in page][0] for q, _, anchor in facts}
for q, _, anchor in facts:
print(f"page {home[q] + 1:2}: {anchor}")page 1: 10,597,729,352
page 41: useful life will change from six years to five years
page 27: positively impacted operating income by $240 million
page 26: reduced sales and marketing costs by $263 million
page 27: $(1.6) billion from our equity investment in RivianNow the closed-book test, with no pages in the prompt:
for question, pattern, answer in facts:
reply, _ = ask(question + " Answer in one short sentence.")
print("right" if re.search(pattern, reply) else "WRONG", "|", reply.strip()[:110])WRONG | As of January 29, 2025, there were approximately 10.7 billion shares of Amazon common stock outstanding.
WRONG | Amazon expects the change to reduce 2025 operating income by approximately $1 billion.
WRONG | Amazon's 2024 annual report has not yet been released, so the specific impact of foreign exchange rate changes
WRONG | Amazon's 2024 Annual Report (Form 10-K) has not yet been released, as the fiscal year 2024 ended on December 3
WRONG | Amazon recorded a $1.4 billion loss on its Rivian investment in the first quarter of 2024.Here, we can see all five answers wrong. The model guessed 10.7 billion shares and a $1.4 billion Rivian loss, and twice it said the report was not out yet. So any right answer later must come from the pages we give it.
We had to swap one fact during this step. Our first pick was Amazon's $2.75 billion Anthropic investment, and the model knew it with no document, so it could not test anything.
How Do We Move a Fact Through a Long Prompt?
Next, we build the stack. We take 30 filler pages from later in the same 10-K. We drop any page that mentions one of our facts, so each fact appears exactly once. Then we insert the fact page at 0%, 25%, 50%, 75% or 100% of the stack. Every page gets a "[Page N]" label, like a real document.
keywords = ["10,597,729,352", "six years to five years", "$240 million", "$263 million", "Rivian"]
filler = [p for p in pages[27:75] if not any(k in p for k in keywords)][:30]
def build(fact_page, position):
docs = filler.copy()
docs.insert(round(position * len(filler)), fact_page)
return "\n\n".join(f"[Page {n + 1}]\n{text}" for n, text in enumerate(docs))
positions = [0, 0.25, 0.5, 0.75, 1.0]
print(len(filler), "filler pages")30 filler pagesDoes the Position of the Fact Matter?
Now, we ask each of the five questions with the fact page at each of the five positions. The question goes after the document. Let's test it:
def run(layout):
hits = {p: 0 for p in positions}
for question, pattern, answer in facts:
for p in positions:
document = build(pages[home[question]], p)
ask_line = question + " Answer in one short sentence."
prompt = (f"{document}\n\n{ask_line}" if layout == "after"
else f"{ask_line}\n\n{document}")
reply, tokens = ask(prompt)
hits[p] += bool(re.search(pattern, reply))
return hits, tokens
after, tokens = run("after")
print(f"prompt size about {tokens} tokens")
print("question after the document: ", " ".join(f"{p:.0%}: {n}/5" for p, n in after.items()))prompt size about 25450 tokens
question after the document: 0%: 5/5 25%: 5/5 50%: 5/5 75%: 5/5 100%: 5/5Here, we can see 25 right answers out of 25. At about 25,450 tokens, Qwen 3.8 found every fact at the start, in the middle and at the end. There is no U shape here.
Should the Question Go Before or After the Document?
In Prompt Caching, we put the question after the document so the cache could be reused. But some people put the question first, so the model knows what to look for while it reads. Let's test that layout too:
before, _ = run("before")
print("question before the document:", " ".join(f"{p:.0%}: {n}/5" for p, n in before.items()))question before the document: 0%: 5/5 25%: 5/5 50%: 5/5 75%: 5/5 100%: 5/5Here, we can see 25 of 25 again. At this length, the place of the question did not change a single answer.
What Happens With a Much Longer Prompt?
25,000 tokens is long, but this model can take far more. So let's make the stack much bigger: every Amazon 10-K page that does not hold one of our facts, plus 30 pages from Alphabet's 10-K.
google = [" ".join(page.extract_text().split()) for page in PdfReader("data/google 10-k 2024.pdf").pages]
amazon_rest = [p for p in pages if not any(k in p for k in keywords)]
filler = amazon_rest + google[:30]
print(len(filler), "filler pages")113 filler pagesThen we run the same five facts at the same five positions, with the question after the document:
long_after, tokens = run("after")
print(f"prompt size about {tokens} tokens")
print("question after the document: ", " ".join(f"{p:.0%}: {n}/5" for p, n in long_after.items()))prompt size about 88862 tokens
question after the document: 0%: 5/5 25%: 5/5 50%: 5/5 75%: 5/5 100%: 5/5Here, we can see 25 of 25 once more, now at about 88,862 tokens. That is more than three times the first stack, and 114 pages from two filings. The fact page was found at the start, in the middle and at the end.
Let me tabulate all three runs for your better understanding:
| Prompt | Question placement | 0% | 25% | 50% | 75% | 100% |
|---|---|---|---|---|---|---|
| About 25,450 tokens | After | 5 of 5 | 5 of 5 | 5 of 5 | 5 of 5 | 5 of 5 |
| About 25,450 tokens | Before | 5 of 5 | 5 of 5 | 5 of 5 | 5 of 5 | 5 of 5 |
| About 88,862 tokens | After | 5 of 5 | 5 of 5 | 5 of 5 | 5 of 5 | 5 of 5 |
Each cell counts the five facts at one position.
How Should We Build a Long Finance Prompt?
Our result is good news: 75 of 75 answers were right across the three runs. But it comes from one model, five facts and short answers. A harder task, like comparing two numbers from far-apart pages, can still go wrong in a long prompt.
So a few habits are still worth keeping:
- Send only the pages that matter. Retrieval in a RAG app should cut the stack before the model sees it, and fewer tokens also mean faster prefill.
- Label every page, like "[Page 27]", so the model can say where a fact came from and we can check it.
- Put the static pages first and the question last. This works well for answers, as we saw here, and for the cache, as we saw in lesson 9.
- Test with your own documents. A closed-book check plus a position test, like this one, takes a few lines of code.
Static parts first, the question last, every page labeled.
Recap
This is how we test a long prompt. We started with the lost in the middle finding from 2023, where models missed facts in the middle of long contexts. We picked five 10-K facts that Qwen 3.8 could not answer from memory, and moved each one from the first page to the last. At about 25,000 tokens, it found all of them at every position, with the question before or after the pages. Finally, at about 89,000 tokens, it still found every fact everywhere, so on this test the middle was not lost.
This is also the end of our Prompt Engineering series. We went from the tokens inside a prompt, through roles, examples, chains, injection, sampling, reasoning, structured output and caching, to long prompts, all on real 10-K filings.
