Module 11: Neural & AI Search Patents

120+ patents covering neural ranking, BERT/Transformers, deep learning IR, embeddings, LLMs, and attention mechanisms.

Overview

Google's search engine has been rebuilt around neural networks. This module covers the patent foundation of AI-driven search — from the original neural ranking experiments through BERT, dense retrieval, large language models, and the emerging AI Overview infrastructure.

Neural Ranking Patents (13 Patents)

US7840569B2 - Neural Network Document Ranking

Year: 2010

Key Innovation: First large-scale application of neural networks to document ranking.

Approach:

Neural network learns ranking features from training data
Inputs: TF-IDF, PageRank, anchor text, query-document features
Output: relevance probability score
Trained on human relevance judgments

US9558233B1 - Selection Quality Scoring

Year: 2015

Key Innovation: User dwell time as a ranking signal.

Signals:

Time between clicking a result and returning to SERP
"Long click" = user found what they needed (positive)
"Short click" = user didn't (negative)
Aggregated across thousands of users per query

US10229166B1 - NavBoost (Implicit User Feedback)

Year: 2019

The Most Important User Signal Patent:

NavBoost Flow:
1. Query performed
2. User clicks result → signal logged
3. Time-on-page measured
4. Return-to-SERP measured
5. Next click (if any) measured
6. Aggregate patterns across users
7. Histogram analysis filters outliers
8. Valid signals re-rank results

Confirmed in 2024 DOJ antitrust trial.

Complete Neural Ranking Patent List

Patent	Title	Key Signal
US7840569B2	Neural Network Ranking	Neural features
US8117209B1	Feature-Based Ranking (Reasonable Surfer)	Click probability
US10229166B1	NavBoost	User behavior
US9558233B1	Selection Quality Score	Dwell time
US9031929B1	Site Quality Score	Site-level quality
US8661029B1	User Feedback Ranking	Explicit feedback
US10296642B1	Engagement Scoring	Engagement depth
US8442984B1	Website Quality Signal	Quality features
US9684697B1	Click Data Quality	Click reliability
US9092510B1	Post-Click Behavior	Return to SERP
US8838587B1	Session Dwell Duration	Session patterns
US11132700B1	A/B Testing Signals	Experiment signals
US8255413B2	Dwell Time Analysis	Time-based quality

BERT / Transformers (18 Patents)

US10452978B2 - Attention Mechanism (Foundation)

Year: 2019

What the Attention Mechanism Does:

Weighs every word against every other word in context
"The bank near the river" vs "deposit at the bank" — context resolves meaning
Self-attention enables deep contextual understanding
Foundation of all modern language model architecture

Query Understanding Impact:

Before BERT: "visa requirements india" → keyword match to "visa" + "india"
After BERT: Understands this is about travel documents, not payment cards,
            and "requirements" signals informational intent for applicants

US20230334045A1 - BERT Query Integration

Year: 2023

Modern BERT Application:

Applied at query time for real-time understanding
Multi-language support (75+ languages in MUM)
Cross-lingual semantic matching
Integrated with entity disambiguation

BERT Patent Reference

Patent	Title	Year
US10452978B2	Attention mechanism (Transformers foundation)	2019
US20230334045A1	BERT query integration	2023
US12111859B2	Cross-attention encoder for ranking	2024
US11782998B2	Dense retrieval with transformers	2022
US20190114362A1	Entity-based embeddings	2019
US10810270B2	Emotional state + browsing history	2020
US20180046834A1	Query-entity matching	2018

Deep Learning Information Retrieval (40+ Patents)

Key Deep Learning IR Concepts

Learned Ranking Functions:

Neural networks learn optimal weighting of ranking signals
Supervised training on human relevance judgments (NDCG optimization)
Gradient boosting + neural features combined
Signal weights are LEARNED, not manually set

Neural Feature Extraction:

Word embeddings capture semantic word relationships
Sentence embeddings capture document meaning
Hierarchical features (word → sentence → document)
Context-dependent feature extraction

Deep Semantic Matching:

Traditional: Count overlapping keywords
Deep Semantic: Compare vector representations
Result: Finds relevant content with zero keyword overlap

Representation Learning:

Documents and queries encoded in shared vector space
Semantic similarity measured by vector distance
Enables approximate nearest-neighbor search at scale

Embeddings & Vector Search (41 Patents)

US12099533B2 - Dense Document Representations

Year: 2022-2024

How Dense Retrieval Works:

Every document encoded as a vector (768-1024 dimensions)
Every query encoded as a vector
Cosine similarity finds best document matches
Can retrieve relevant docs without keyword overlap

SEO Implication: Content that uses entirely different vocabulary but covers the same concept can rank. Semantic coverage > keyword density.

US11782998B2 - Vector Similarity Search

Year: 2022

Key Innovation: Efficient approximate nearest-neighbor search in high-dimensional space.

Practical Impact:

"People also ask" questions found via vector similarity
Related content recommendations
Semantic duplicate detection
Cross-language content matching

Key Embedding Patents

Patent	Innovation
US12099533B2	Dense document representations
US11782998B2	Vector similarity search
US20190114362A1	Entity embeddings
US12210825B2	Image captioning embeddings
US12051205B1	Multimodal embeddings

LLMs for Search (29 Patents)

US11769017B1 - PaLM/LaMDA for Generative Search

Year: 2023

Application:

Generative answers to conversational queries
Synthesizes information from multiple documents
Grounded in retrieved passages (RAG architecture)
Powers AI Overviews (formerly Search Generative Experience)

US11003865B1 - Retrieval-Augmented Generation (RAG)

Year: 2021

RAG Architecture:

1. User query received
2. Query used to retrieve relevant documents from index
3. Retrieved documents passed to LLM as context
4. LLM generates answer grounded in retrieved docs
5. Answer citations linked to source documents
6. Displayed as AI Overview

Why This Matters for SEO:

Your page must be RETRIEVABLE (traditional SEO)
Your content must be EXTRACTABLE (clear, direct answers)
Your claims must be VERIFIABLE (citeable with primary sources)

US12353469B1 - LLM Output Verification

Year: 2024

Verification Flow:

LLM generates claim: "X causes Y"
System searches retrieval index for supporting evidence
Evidence confidence scored (0-1)
Claims below confidence threshold filtered or flagged
High-confidence claims shown in AI Overview with citation

For AI Overview inclusion: Be the most clearly cited, primary source for your claims.

Attention Mechanisms & NLP

How Attention Works in Ranking

SEO Implications for Neural Search

1. Write for Semantic Meaning, Not Keywords

Traditional approach: "Use keyword X every 200 words" Neural approach: Ensure your content conveys the CONCEPT thoroughly

2. Clear Direct Answers for RAG Extraction

BAD (for RAG):  "There are many factors to consider when..."
GOOD (for RAG): "The three primary causes of X are Y, Z, and W."

3. Build Entity Co-Occurrence

Entity embeddings drive retrieval. Ensure:

Primary entity clearly identified in first paragraph
Related entities mentioned in context
Entity relationships explicitly stated (not implied)

4. Cite Verifiable Sources

Per US12353469B1 (LLM verification):

Every factual claim should have a citable source
Government, academic, and institutional sources preferred
Statistics should link to primary research

5. Multimodal Content

US12051205B1 shows Google combining text + image + video signals:

Add descriptive alt text to all images
Include transcripts for video content
Use structured data to connect media to content topics

Key Patents Referenced

Patent	Title	Year
US7840569B2	Neural Network Document Ranking	2010
US10452978B2	Attention Mechanism (BERT Foundation)	2019
US10229166B1	NavBoost	2019
US11003865B1	RAG	2021
US12099533B2	Dense Document Representations	2022-2024
US12353469B1	LLM Output Verification	2024
US12051205B1	Multimodal Embeddings	2023

Next Steps

Voice Search Module — Conversational search
User Behavior Module — CTR and engagement
Comprehensive Expansion — 2024-2025 developments

Module 11: Neural & AI Search Patents ​

Overview ​

Neural Ranking Patents (13 Patents) ​

US7840569B2 - Neural Network Document Ranking ​

US9558233B1 - Selection Quality Scoring ​

US10229166B1 - NavBoost (Implicit User Feedback) ​

Complete Neural Ranking Patent List ​

BERT / Transformers (18 Patents) ​

US10452978B2 - Attention Mechanism (Foundation) ​

US20230334045A1 - BERT Query Integration ​

BERT Patent Reference ​

Deep Learning Information Retrieval (40+ Patents) ​

Key Deep Learning IR Concepts ​

Embeddings & Vector Search (41 Patents) ​

US12099533B2 - Dense Document Representations ​

US11782998B2 - Vector Similarity Search ​

Key Embedding Patents ​

LLMs for Search (29 Patents) ​

US11769017B1 - PaLM/LaMDA for Generative Search ​

US11003865B1 - Retrieval-Augmented Generation (RAG) ​

US12353469B1 - LLM Output Verification ​

Attention Mechanisms & NLP ​

How Attention Works in Ranking ​

SEO Implications for Neural Search ​

1. Write for Semantic Meaning, Not Keywords ​

2. Clear Direct Answers for RAG Extraction ​

3. Build Entity Co-Occurrence ​

4. Cite Verifiable Sources ​

5. Multimodal Content ​

Key Patents Referenced ​

Next Steps ​