Seminar

PhD Seminar • Data Systems • Semantic Table Discovery in Model Lakes: A Benchmark

Joe Petrik — Thu, 10 Jul 2025 22:07:57 +0000

PhD Seminar • Data Systems • Semantic Table Discovery in Model Lakes: A Benchmark Joe Petrik Thu, 07/10/2025 - 18:07

Please note: This PhD seminar will take place in DC 3301.

Zhengyuan Dong, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Renée J. Miller

Model Lakes are emerging large-scale repositories of machine learning artifacts. Although they greatly facilitate model sharing, discovery still relies on keyword or full-text search over textual metadata, which overlooks the rich, structured information — especially performance and configuration tables-embedded in model reports.

In this work, we advance model discovery by leveraging table-discovery techniques within Model Lakes. We first formalize a novel ground-truth methodology for model relatedness, based on three complementary signals: explicit references in model cards, citation links among associated papers, and shared training datasets. We then build and publicly release a benchmark over 100 K Hugging Face models, extracting every table from model cards, GitHub READMEs, arXiv preprints, and BibTeX entries. Compared to standard data-lake tables, our tables are smaller but exhibit far denser inter-table relationships, reflecting the tight coupling of model evolution. To retrieve related models, we adapt a canonical Data Lake task, unionable table search, and compare against dense and sparse IR baselines.

Our union-based semantic search achieves 54.8% P@1 overall (54.7% on paper-citation ground truth, 30.8% on model-card inheritance, 30.2% on shared-dataset signals), while simple metadata retrieval peaks at 36.8% P@1. Denser citation-graph edges boost precision to 74.8%, and a header-value concatenation augmentation raises overall P@1 to 60.3%. To our knowledge, this is the first empirical study applying Data Lake management principles to Model Discovery using large-scale real-world machine learning artifacts. By demonstrating that structured table information uncovers deep model relationships, we lay the groundwork for more accurate retrieval, systematic comparison, and seamless integration of models within Model Lakes.

Seminar • Algorithms and Complexity • Closure Results for Polynomial Factorization and Some Applications

Joe Petrik — Thu, 10 Jul 2025 18:59:16 +0000

Seminar • Algorithms and Complexity • Closure Results for Polynomial Factorization and Some Applications Joe Petrik Thu, 07/10/2025 - 14:59

Please note: This seminar will take place in DC 1304 and online.

Shubhangi Saraf, Associate Professor
Departments of Mathematics and Computer Science, University of Toronto

I will talk about a recent result showing that algebraic formulas and constant-depth circuits are closed under taking factors. In other words, the complexity of factors of polynomials computable by algebraic formulas or constant depth algebraic circuits is not much more than the complexity of the original polynomial itself.

This result turns out to be an elementary consequence of a fundamental and surprising result of Furstenberg from the 1960s, which gives a non-iterative description of the power series roots of a bivariate polynomial. Combined with standard structural ideas in algebraic complexity, we observe that this theorem yields the desired closure results. We will see applications of this result to deterministic algorithms for factoring, hardness/randomness tradeoffs, as well as GCD computation of polynomials.

This talk is based on joint works with Somnath Bhattacharjee, Mrinal Kumar, Shanthanu Rai, Varun Ramanathan and Ramprasad Saptharishi.

To attend this seminar in person, please go to DC 1304. You can also attend virtually on Zoom.

PhD Seminar • Machine Learning | Information Retrieval • Modern IR Evaluation in the Retrieval Augmented Generation (RAG) Era

Joe Petrik — Tue, 08 Jul 2025 17:42:14 +0000

PhD Seminar • Machine Learning | Information Retrieval • Modern IR Evaluation in the Retrieval Augmented Generation (RAG) Era Joe Petrik Tue, 07/08/2025 - 13:42

Please note: This PhD seminar will take place in DC 3301.

Nandan Thakur, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Jimmy Lin

Traditional IR evaluation (e.g., TREC, Cranfield paradigm) constructs test collections that use fixed corpora and pool relevance judgments, a practice that minimally captures the challenges of RAG applications. This talk starts by mentioning limitations in prevalent IR benchmarks, comprising either stale data, incomplete labels, or simplistic queries. In particular, we motivate why retrieval evaluation must evolve and why a metric shift is needed for IR evaluation in modern-day systems as an emerging requirement. We survey FreshStack, a holistic benchmark that addresses these gaps, by constructing test collections with recent StackOverflow Q&As and GitHub documents to reflect real-world programming questions, providing insight on the diversity-focused metrics used in IR evaluation. The goal is to give practitioners insights into the limits of traditional IR evaluation and guide them toward more realistic, robust evaluation practice of IR systems in the modern-day RAG applications.

Seminar • Algorithms and Complexity • Synthesis and Arithmetic of Quantum Circuits

Joe Petrik — Fri, 27 Jun 2025 17:19:50 +0000

Seminar • Algorithms and Complexity • Synthesis and Arithmetic of Quantum Circuits Joe Petrik Fri, 06/27/2025 - 13:19

Please note: This seminar will take place on DC 1304 and online.

Amolak Ratan Kalra, PhD candidate
Institute for Quantum Computing, University of ��ݮ��Ƶ

Efficient decomposition of a unitary operator U using words from a universal gate set G is a fundamental problem in quantum computing. The process by which this is achieved is called circuit synthesis. This problem arises naturally in the context of quantum circuit compilation.

In this talk, I will introduce this problem and explain how one can use tools from number theory to solve it. I will then explain some of our more recent results that build on this connection.

To attend this seminar in person, please go to DC 1304. You can also attend virtually on Zoom.

PhD Seminar • Systems and Networking • Dynamic SLA-aware Network Slice Monitoring with Programmable Data Planes

Joe Petrik — Tue, 24 Jun 2025 01:06:33 +0000

PhD Seminar • Systems and Networking • Dynamic SLA-aware Network Slice Monitoring with Programmable Data Planes Joe Petrik Mon, 06/23/2025 - 21:06

Please note: This PhD seminar will take place in DC 1304.

Niloy Saha, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Raouf Boutaba

Next generation networks increasingly rely on network slices — logical networks tailored to specific application requirements, each with distinct Service-Level Agreements (SLAs). Ensuring compliance with these SLAs requires continuous, real-time monitoring of end-to-end performance metrics for each slice, within a limited telemetry budget. However, existing monitoring solutions based on sketches or probabilistic sampling lack end-to-end visibility and treat all traffic uniformly. This leads to inaccurate monitoring of critical slices in order to stay within budget. We present SliceScope, a slice-aware telemetry system that dynamically allocates monitoring resources across a diverse set of slices, based on SLA criticality and evolving network conditions.

SliceScope combines: (1) a data-plane primitive that enables per-packet end-to-end visibility for each slice with tunable accuracy-overhead trade-off, and (2) a control strategy that adjusts this trade-off per-slice to allocate limited telemetry budget where it matters most. Our evaluation results, conducted on a testbed with programmable switches and in large-scale simulations with a mixture of different slice types, demonstrate that SliceScope provides real-time, fine-grained monitoring of per-slice metrics, and tracks critical slices up to 4× more accurately compared to static or SLA-agnostic baselines.

PhD Seminar • Machine Learning | Deep Learning • Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization

Joe Petrik — Fri, 13 Jun 2025 01:19:23 +0000

PhD Seminar • Machine Learning | Deep Learning • Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization Joe Petrik Thu, 06/12/2025 - 21:19

Please note: This PhD seminar will take place online.

Aref Jafari, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Ali Ghodsi

Knowledge Distillation (KD) has been extensively used for natural language understanding (NLU) tasks to improve a small model’s (a student) generalization by transferring the knowledge from a larger model (a teacher). Although KD methods achieve state-of-the-art performance in numerous settings, they suffer from several problems limiting their performance. It is shown in the literature that the capacity gap between the teacher and the student networks can make KD ineffective. Additionally, existing KD techniques do not mitigate the noise in the teacher’s output: modeling the noisy behaviour of the teacher can distract the student from learning more useful features.

We propose a new KD method that addresses these problems and facilitates the training compared to previous techniques. Inspired by continuation optimization, we design a training procedure that optimizes the highly non-convex KD objective by starting with the smoothed version of this objective and making it more complex as the training proceeds. Our method (Continuation-KD) achieves state-of-the-art performance across various compact architectures on NLU (GLUE benchmark) and computer vision tasks (CIFAR-10 and CIFAR- 100).

Attend this PhD seminar virtually on MS Teams.

PhD Seminar • Artificial Intelligence • Towards Cost-Effective Reward Guided Text Generation

Joe Petrik — Tue, 10 Jun 2025 14:51:54 +0000

PhD Seminar • Artificial Intelligence • Towards Cost-Effective Reward Guided Text Generation Joe Petrik Tue, 06/10/2025 - 10:51

Please note: This PhD seminar will take place in DC 2584 and online.

Ahmad Rashid, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Pascal Poupart

Reward-guided text generation (RGTG) has emerged as a viable alternative to offline reinforcement learning from human feedback (RLHF). RGTG methods can align baseline language models to human preferences without further training as in standard RLHF methods. However, they rely on a reward model to score each candidate token generated by the language model at inference, incurring significant test-time overhead. Additionally, the reward model is usually only trained to score full sequences, which can lead to sub-optimal choices for partial sequences.

In this work, we present a novel reward model architecture that is trained, using a Bradley-Terry loss, to prefer the optimal expansion of a sequence with just a single call to the reward model at each step of the generation process. That is, a score for all possible candidate tokens is generated simultaneously, leading to efficient inference. We theoretically analyze various RGTG reward models and demonstrate that prior techniques prefer sub-optimal sequences compared to our method during inference. Empirically, our reward model leads to significantly faster inference than other RGTG methods. It requires fewer calls to the reward model and performs competitively compared to previous RGTG and offline RLHF methods.

To attend this PhD seminar in person, please go to DC 2584. You can also attend virtually on Zoom.

PhD Seminar • Algorithms and Complexity • An Improved Fully Dynamic Algorithm for Counting 4-Cycles in General Graphs

Joe Petrik — Fri, 06 Jun 2025 18:22:12 +0000

PhD Seminar • Algorithms and Complexity • An Improved Fully Dynamic Algorithm for Counting 4-Cycles in General Graphs Joe Petrik Fri, 06/06/2025 - 14:22

Please note: This PhD seminar will take place in DC 1304 and online.

Vihan Shah, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Sepehr Assadi

We study subgraph counting over fully dynamic graphs, which undergo edge insertions and deletions. Maintaining the number of triangles in fully dynamic graphs is very well studied and has an upper bound of O(m^{1/2}) for the update time [KNN+20]. There is also a conditional lower bound of approximately Omega(m^{1/2}) for the update time [HKNS15] under the OMv conjecture implying that Theta(m^{1/2}) is the “right answer” for the update time of counting triangles. More recently, [HHH22] studied the problem of maintaining the number of 4-cycles in fully dynamic graphs and designed an algorithm with O(m^{2/3}) update time which is a natural generalization of the approach for counting triangles. Thus, it seems natural that O(m^{2/3}) might be the correct answer for the complexity of the update time for counting 4-cycles.

In this work, we present an improved algorithm for maintaining the number of 4-cycles in fully dynamic graphs. Our algorithm achieves a worst-case update time of O(m^{2/3-eps}) for some constant eps>0. Our approach crucially uses fast matrix multiplication and leverages recent developments therein to get an improved runtime. Using the current best value of the matrix multiplication exponent omega=2.371339 we get eps=0.009811 and if we assume the best possible exponent i.e., omega=2 then we get eps=1/24. The lower bound for the update time is Omega(m^{1/2}), so there is still a big gap between the best-known upper and lower bounds. The key message of our paper is demonstrating that O(m^{2/3}) is not the correct answer for the complexity of the update time.

To attend this PhD seminar in person, please go to DC 1304. You can also attend virtually on Zoom.

Seminar • Algorithms and Complexity • Linear Hashing Is Optimal

Joe Petrik — Fri, 06 Jun 2025 13:14:14 +0000

Seminar • Algorithms and Complexity • Linear Hashing Is Optimal Joe Petrik Fri, 06/06/2025 - 09:14

Please note: This seminar will take place in DC 1304 and online.

Vinayak Kumar, PhD student
Computer Science Theory Group, UT Austin

When n balls are independently and uniformly tossed into n bins, the expected max-load — the number of balls in the heaviest bin — is O(logn/loglogn). This classical result plays a central role in the analysis of hashing with chaining and load balancing. However, implementing a truly random hash function is often impractical due to its high computational and storage costs.

In this talk, I will present a recent result showing that hashing n balls into n bins via a random matrix over F2 achieves the same expected max-load of O(logn/loglogn). This simple and efficient hash family matches the performance of a fully random function and resolves an open question posed by Alon, Dietzfelbinger, Miltersen, Petrank, and Tardos.

Based on joint work with Michael Jaber and David Zuckerman.

To attend this seminar in person, please go to DC 1304. You can also attend virtually on Zoom.

PhD Seminar • Data Systems • UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction

Joe Petrik — Tue, 03 Jun 2025 14:33:16 +0000

PhD Seminar • Data Systems • UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction Joe Petrik Tue, 06/03/2025 - 10:33

Please note: This PhD seminar will take place in DC 3301.

Xiangru Jian, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Tamer Özsu

Autonomous agents that navigate Graphical User Interfaces (GUIs) to automate tasks like document editing and file management can greatly enhance computer workflows. While existing research focuses on online settings, desktop environments, critical for many professional and everyday tasks, remain underexplored due to data collection challenges and licensing issues.

We introduce UI-Vision, the first comprehensive, license-permissive benchmark for offline, fine-grained evaluation of computer use agents in real-world desktop environments. Unlike online benchmarks, UI-Vision provides: (i) dense, high-quality annotations of human demonstrations, including bounding boxes, UI labels, and action trajectories (clicks, drags, and keyboard inputs) across 83 software applications, and (ii) three fine-to-coarse grained tasks — Element Grounding, Layout Grounding, and Action Prediction — with well-defined metrics to rigorously evaluate agents’ performance in desktop environments. Our evaluation reveals critical limitations in state-of-the-art models like UI-TARS-72B, including issues with understanding professional software, spatial reasoning, and complex actions like drag-and-drop. These findings highlight the challenges in developing fully autonomous computer-use agents. With UI-Vision, we aim to advance the development of more capable agents for real-world desktop tasks.