Ziwei Fan

Portrait of Ziwei Fan

Ziwei Fan

ML Research Engineer at Apple  ·  Ph.D. in Computer Science, University of Illinois Chicago

I build intelligent systems for personalization & recommendation and code migration & translation — including personalized ranking models, multi-modal LLM fine-tuning, agentic test generation, and repo-scale code documentation.

My research centers on personalized generation & retrieval and large language models (LLMs) for code. I design personalized ranking models that address data sparsity, representation collapse, and cross-domain shift in recommender systems, and I build LLM agents that migrate and translate code at repository scale while aligning multi-modal LLMs with real-world tail signals. More recently, I explore proactive, just-in-time agents that infer when to act and re-rank content on-device, without an explicit request.

Proactive Just-in-Time Agents

On-device agents that decide when to act, what to ask, and how to re-rank — as in Kairosrank.

Personalized Ranking Models

Sequential and graph-based ranking for large-scale recommendation and user modeling.

LLM for Code

Self-debugging, agentic test generation, and evaluation for code translation & migration — shipping in AWS Transform for Mainframe.

Representation & Diversity

Mitigating representation collapse; controllable diversification for recommendation.

02 News

2026
Released Kairosrank, a side project — a proactive, just-in-time on-device re-ranking agent (browser extension) that decides when to act, what to ask, and how to re-rank (code · tech report).
2026
New preprint on k-token merging in the latent embedding space for LLM sequence compression (arXiv 2604.15153).
2025
Preprint “Lossless Token Sequence Compression via Meta-Tokens” released (arXiv 2506.00307).
2024
Two papers accepted at KDD 2024 (LLM reasoning & planning for code development; pre-training with transferable attention for cross-market sequential recommendation) and one at NAACL 2024 (EIVEN, multi-modal LLM for implicit attribute value extraction).
2023 · Nov
AWS Clean Rooms ML for audience expansion launched at re:Invent 2023, powered by transformer user-sequence embeddings.
2023 · Jul
SIGIR ’23 paper on graph collaborative-signal denoising & augmentation received the Best Short Paper Award (Honorable Mention) 🏅.
2023 · May
Defended my Ph.D. thesis — Data Sparsity in Recommender Systems: Collaborative Neighbors Enrichment. Joined Amazon as Applied Scientist.
2023
Papers accepted at UAI ’23, WWW ’23, SIGIR ’23, and CIKM ’23.

03 Experience

Jul 2026 — Present

ML Research Engineer

Apple

May 2023 — Jul 2026

Applied Scientist II

Amazon

  • Prime Video Homepage & Notifications (RecSys). Homepage ranking model product delivery with positive stream-hour lift. Bayesian day-of-week / hour-of-day optimization for in-app notifications with additional WW stream-hour gain.
  • AWS Transform for Mainframe — Mainframe / Java code migration agent (CodeLLM). KDD ’24
    • Agentic Automatic Test Generation for system-level functional-equivalence testing with differential fuzzing; condition/exception execution-path extraction via AST + LLM improves over strong random-testing baselines.
    • Agentic Code Documentation for long (repo-level, 20K+ LoC per file) context extraction with test-time context optimization for hallucination reduction.
    • LLM-as-Judge Ensemble for quality measurement of generated documents via perturbations and claim-level metric definitions.
    • LLM post-training for prompt compression via adaptive meta-token fine-tuning. arXiv ’25arXiv ’26
    • Self-debug+ agent with format verifier — measurable success-rate@1 improvement for Java Upgrade.
  • AWS Clean Rooms ML — Audience Expansion (RecSys). Transformer user-sequence embeddings and seed-user expansion for ads campaigns. Launched at re:Invent 2023.
  • Multi-modal LLM fine-tuning. Implicit attribute-value extraction (EIVEN) with tail-attribute improvements. NAACL ’24
  • Personalized recommendation explanation. Aspect-instructed, logic-scaffolded explanation generation with LLMs. WSDM ’24
2020 — 2022

Research / Applied Scientist / Data Scientist Interns

Salesforce Research · AWS AI · Spotify Research · Stitch Fix

  • Product knowledge-graph pre-training for zero-shot item-based recommendation (Salesforce Research). CIKM ’23
  • Personalized federated domain adaptation for item-to-item recommendation (AWS AI). UAI ’23
  • Discovery-episode ranking via multi-source augmentations (Spotify Research). arXiv ’23
  • Data science for personalized styling (Stitch Fix).
Aug 2018 — May 2023

Graduate Research & Teaching Assistant

University of Illinois Chicago

  • Transformer encoders for sequential recommendation. WWW ’22CIKM ’21BigData ’22
  • Data augmentation & denoising for long-tail recommendation. SIGIR ’21SIGIR ’23
  • Representation collapse, diversity, and robustness. WWW ’23arXiv ’24
  • Federated / transfer learning for cross-domain recommendation. KDD ’24

04 Education

Ph.D., Computer Science

University of Illinois Chicago

Aug 2018 — May 2023

Thesis: Data Sparsity in Recommender Systems — Collaborative Neighbors Enrichment. Advised by Philip S. Yu.

M.S., Computer Science

Purdue University — Indianapolis (IUPUI)

Aug 2016 — May 2018

B.Eng., Network Engineering

South China Agricultural University

Sep 2012 — Jun 2016

05 Selected Publications

Full list on Google Scholar →
Tech
Report
2026

Kairosrank: A Proactive, Just-in-Time Agent That Decides When to Break Silence, What to Ask, and How to Re-Rank

Ziwei Fan.

R&D prototype PDF Code
arXiv2026

Compressing Sequences in the Latent Embedding Space: k-Token Merging for Large Language Models

Zihao Xu, John Harvill, Ziwei Fan, Yizhou Sun, Hao Ding, Hao Wang.

arXiv2025

Lossless Token Sequence Compression via Meta-Tokens

John Harvill, Ziwei Fan, Hao Wang, Yizhou Sun, Hao Ding, Luke Huan, Anoop Deoras.

KDD2024

Reasoning and Planning with Large Language Models in Code Development

Hao Ding, Ziwei Fan, Ingo Guehring, Gaurav Gupta, Wooseok Ha, Jun Huan, Linbo Liu, Behrooz Omidvar-Tehrani, Shiqi Wang, Hao Zhou.

KDD2024

Pre-training with Transferable Attention for Addressing Market Shifts in Cross-Market Sequential Recommendation

Chen Wang, Ziwei Fan, Liangwei Yang, Mingdai Yang, Xiaolong Liu, Zhiwei Liu, Philip S. Yu.

NAACL2024

EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM

Henry Peng Zou, Gavin Heqing Yu, Ziwei Fan, Bu Dan, Han Liu, Peng Dai, Dongmei Jia, Cornelia Caragea.

WSDM2024

Logic-Scaffolding: Personalized Aspect-Instructed Recommendation Explanation Generation using LLMs

Behnam Rahdari, Hao Ding, Ziwei Fan, Yifei Ma, Zhuotong Chen, Branislav Kveton, Anoop Deoras.

arXiv2024

Sequential Recommendation with Controllable Diversification: Representation Degeneration and Diversity

Ziwei Fan, Zhiwei Liu, Hao Peng, Philip S. Yu.

UAI2023

Personalized Federated Domain Adaptation for Item-to-Item Recommendation

Ziwei Fan, Nghia Hoang, Hao Ding, Anoop Deoras.

SIGIR2023

Graph Collaborative Signals Denoising and Augmentation for Recommendation

Ziwei Fan, Ke Xu, Zhang Dong, Hao Peng, Jiawei Zhang, Philip S. Yu.

Best Short Paper (Honorable Mention) PDF Code
WWW2023

Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation

Ziwei Fan, Zhiwei Liu, Hao Peng, Philip S. Yu.

CIKM2023

Zero-Shot Item-Based Recommendation via Multi-Task Product Knowledge Graph Pre-training

Ziwei Fan, Zhiwei Liu, Shelby Heinecke, Jianguo Zhang, Huan Wang, Caiming Xiong, Philip S. Yu.

arXiv2023

Episodes Discovery Recommendation with Multi-Source Augmentations

Ziwei Fan, Alice Wang, Zahra Nazari.

Spotify Research internship PDF
BigData2022

Sequential Recommendation with Auxiliary Item Relationships via Multi-Relational Transformer

Ziwei Fan, Zhiwei Liu, Chen Wang, Peijie Huang, Hao Peng, Philip S. Yu.

WWW2022

Sequential Recommendation via Stochastic Self-Attention

Ziwei Fan, Zhiwei Liu, Alice Wang, Zahra Nazari, Lei Zheng, Hao Peng, Philip S. Yu.

CIKM2021

Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer

Ziwei Fan*, Zhiwei Liu*, Jiawei Zhang, Yun Xiong, Lei Zheng, Philip S. Yu.

CIKM2021

Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation

Ziwei Fan, Zhiwei Liu, Shen Wang, Lei Zheng, Philip S. Yu.

Best Short Paper Nomination PDF Code
SIGIR2021

Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-Training Transformer

Ziwei Fan*, Zhiwei Liu*, Yu Wang, Philip S. Yu.