ChrisMcCormickAI
Weekly Research Group, June 24th, 2021
This week, Nick gave us an overview of some of the leading Retriever models out there. We focused primarily on Generative models (which produce novel text as answers, rather than returning a “span” of reference text), including BART and T5. We also looked at end-to-end systems like RAG and the recent “Fusion in Decoder”.
You can view our notes from the discussion here: https://docs.google.com/document/d/1NGBtUurxT4COhbq_2g50YZxqmTNJPtsoYxC-FJcUUKs/edit#heading=h.60i9021hvriw
This forum post serves as a place for further discussion and questions from the session: https://discourse.chrismccormick.ai/t/chatbots-conversational-ai-discussion-session-ep-3-reader-options/97
Next week, we’ll be switching back to looking at techniques for incorporating categorical and numerical features into BERT. I’ll share the results of some experiments on last week’s Clothing Review dataset (using the simple unimodal strategy), as well as what I’ve learned so far about some of the more complex techniques for incorporating other features. Sign up here:
https://www.chrismccormick.ai/weekly-discussion-group
You can also sign up for our weekly newsletter here: https://www.chrismccormick.ai/subscribe