Projects for Volunteers at Red Hen Lab

Most work in Red Hen Lab is done by volunteers who want the experience of working in a large research group with sophisticated projects. If you are interested, please write to redhenlab@gmail.com. Below are some of the projects for which we seek volunteers. A more extended list is at the Barnyard of Possible Specific Projects.

Data Capture

Talent Scout

Mentor: Michael Schoop and the Red Hen Lab team. Default but negotiable size: medium 175 hour project. Difficulty: MEDIUM. Coders would need to work remotely inside the Case Western Reserve University High Performance Computing Center so as to have adequate hardware resources. Study https://sites.google.com/case.edu/techne-public-site/cwru-hpc-orientation. Other skills: basic *nix abilities, Python.


This project intends to create an AI assistant, named TalentScout, for students, universities, government organizations, NGOs, businesses (including not-for-profit and for-profit) that would help with internships. Students need to be located and matched with organizations. Organizations need help designing and conducting internship programs. Everyone in the entire internship enterprise needs help. This initial project would be restricted to the Cleveland area, and more generally to Northeast Ohio, but it could become a prototype that would inspire similar open-source AI Talent Scouts in other areas. An open-source foundation model would be the basis, but it would be greatly built out into a Retrieval-Augmented-Generation system that relies on massive stores of documents and data from everyone involved in Northeast Ohio. One of the partners in this initiative providing guidance is the Greater Cleveland Partnership, which is the largest chamber of commerce in the USA. The initial TalentScout would probably be built inside CWRU HPC, but some of the recently-developed open-source foundation models can be run on devices in the high-end consumer range. In principle, such a device could also be a web server. A registration system could be established to control access to some extent. Accordingly, a Red Hen Lab - Greater Cleveland Partnership service available to the appropriate community in Northeast Ohio might be possible. The task is to design TalentScout and build its prototype.


There will be many modules in this overarching project and we invite candidates to develop proposals for each of the modules below. 


Skills required: Experience with LLM design.


Frame Blender

Mentor: Wenyue Suzie Xi, wenyue.sxi@gmail.com, and team. Default but negotiable size: medium 175 hour project. Difficulty: MEDIUM-HARD. Coders would need to work remotely inside the Case Western Reserve University High Performance Computing Center so as to have adequate hardware resources. Study https://sites.google.com/case.edu/techne-public-site/cwru-hpc-orientation. Other skills: basic *nix abilities, Python.


This project intends to use open-source GenAI models to generate frame blends. Techniques include installing the basic model, fine-tuning, Retrieval-Augmented-Generation, prompt engineering, chain-of-thought, and causal inference. introduction: after inputting the FrameNet xml data of Cause_Motion, Judgment, and Communication, the following example is a possible frame blending case generated by ChatGPT: 


"The judge's ruling pushed the defendant towards a new trial."

"Her criticism drove the conversation into deeper introspection."

"The leader's decision propelled the company towards innovative strategies."

"His refusal nudged the team away from the conventional approach."

"The teacher's encouragement steered the student towards academic excellence."

"The critic's harsh words thrust the artist into the spotlight of controversy."

"The mentor's advice guided her thoughts towards a more positive outlook."

"The jury's verdict sent the community into a state of unrest."

"The coach's strategy shifted the team's focus towards defensive plays."

"The therapist's insights led the patient into a journey of self-discovery.”


The tasks include developing the project infrastructure, implementing and examining different methods of prompt engineering, defining the measuring metrics, and evaluating the performance of various methods/models with statistical results. This journey requires both the literature review for exploring methods and hands-on coding for implementing the methods, plus some statistical experiments to evaluate the effectiveness of the proposed methods. This project will be valuable for those who are interested in Large Language Models and Natural Language Processing with solid coding skills. The ideal proposal should demonstrate your understanding of the FrameNet dataset and multiple LLMs(their advantages and limitations), and it’s also helpful to read (and potentially implement some simple tasks) more about chain-of-thought, prompt engineering, and frame blending. This project is an open-ended exploratory process, and it’s exciting to push forward the study of frame blending in this LLMs era with collective effort.  


You would build upon existing Red Hen Lab products. See:



The following are some related references: 

FrameNet https://framenet.icsi.berkeley.edu/framenet_data 

ChatGPT https://openai.com/blog/chatgpt 

Llama2 https://huggingface.co/docs/transformers/main/model_doc/llama2 

Chain-of-thought  https://arxiv.org/abs/2201.11903 

Prompt-engineering https://github.com/thunlp/PromptPapers


There are several independent but supplementary projects that could be designed under the FrameBlender umbrella. 



Chatty AI for Frames and Construction Grammar

Mentor: Mark Turner (turner@case.edu) and team.  Coders would need to work inside the Case Western Reserve University High Performance Computing Center so as to have adequate hardware resources. Study https://sites.google.com/case.edu/techne-public-site/cwru-hpc-orientation .  Skills include working inside CWRU HPC (study the site for specifics), the ability to use standard Linux commands to interact with Red Hen Lab's vast data set, standard techniques of machine learning for fine-tuning an open-source foundation model (such as LaMDA, OpenAssistant, etc.). For a guide to such machine learning skills, ask Turner for a copy of Copilots for Linguists: AI, Constructions, and Frames (Cambridge University Press, 2024).

For the background on this project, on which you would be building, study:

There are several independent but supplementary projects that could be designed under the Chatty AI umbrella. 

Propose a systemlike FrameChat, but that can handle multimodal data other than text, on the basis of the articles and services above.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<?xml-stylesheet type="text/xsl" href="frame.xsl"?>

<frame cBy="ChW" cDate="02/07/2001 04:12:10 PST Wed" name="Cause_motion" ID="55" xsi:schemaLocation="../schema/frame.xsd" xmlns="http://framenet.icsi.berkeley.edu" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

    <definition>&lt;def-root&gt;An &lt;fen&gt;Agent&lt;/fen&gt; causes a &lt;fen&gt;Theme&lt;/fen&gt; to move from a &lt;fen&gt;Source&lt;/fen&gt;, along a &lt;fen&gt;Path&lt;/fen&gt;, to a &lt;fen&gt;Goal&lt;/fen&gt;.  Different members of the frame emphasize the trajectory to different degrees, and a given instance of the frame will usually leave some of the &lt;fen&gt;Source&lt;/fen&gt;, &lt;fen&gt;Path&lt;/fen&gt; and/or &lt;fen&gt;Goal&lt;/fen&gt; implicit. The completion of motion is not required (unlike the Placing frame, see below), although individual sentences annotated with this frame may emphasize the &lt;fen&gt;Goal&lt;/fen&gt;.  

&lt;ex&gt;&lt;/ex&gt;

This frame is very broad and contains several different kinds of words that refer to causing motion.  Some words in this frame do not emphasize the &lt;fen&gt;Manner&lt;/fen&gt;/&lt;fen&gt;Means&lt;/fen&gt; of causing the motion (transfer.v, move.v).  For many of the others (cast.v, throw.v, chuck.v, etc.), the &lt;fen&gt;Agent&lt;/fen&gt; has control of the &lt;fen&gt;Theme&lt;/fen&gt; only at the &lt;fen&gt;Source&lt;/fen&gt; of motion, and does not experience overall motion.  For others (e.g. drag.v, push.v, shove.v, etc.) the &lt;fen&gt;Agent&lt;/fen&gt; has control of the &lt;fen&gt;Theme&lt;/fen&gt; throughout the motion; for these words, the &lt;fen&gt;Theme&lt;/fen&gt; is resistant to motion due to some friction with the surface along which they move.  

&lt;ex&gt;&lt;fex name="Agent"&gt;She&lt;/fex&gt; &lt;t&gt;threw&lt;/t&gt; &lt;fex name="Theme"&gt;her shoes&lt;/fex&gt; &lt;fex name="Goal"&gt;into the dryer&lt;/fex&gt; .&lt;/ex&gt;

&lt;ex&gt;&lt;fex name="Agent"&gt;The mechanic&lt;/fex&gt; &lt;t&gt;dragged&lt;/t&gt; &lt;fex name="Theme"&gt;the jack&lt;/fex&gt; &lt;fex name="Source"&gt;out from under the car&lt;/fex&gt; .&lt;/ex&gt;

&lt;ex&gt;&lt;fex name="Agent"&gt;We&lt;/fex&gt; will &lt;t&gt;move&lt;/t&gt; &lt;fex name="Theme"&gt;the sofa&lt;/fex&gt; &lt;fex name="Source"&gt;out of the room&lt;/fex&gt; &lt;fex name="Path"&gt;through the french doors&lt;/fex&gt;, &lt;fex name="Path"&gt;down the stairs&lt;/fex&gt;, and &lt;fex name="Goal"&gt;onto the sidewalk&lt;/fex&gt; .&lt;/ex&gt;

&lt;ex&gt;&lt;/ex&gt;

&lt;ex&gt;&lt;/ex&gt;

Have a look at https://framenet.icsi.berkeley.edu/frameIndex to get oriented.

The project is to train, refine, and deploy a chatbot on all things having to do with Construction Grammar and FrameNet that could hold conversations with interested parties, explaining subjects, giving directions to resources, etc. Of course, this chatbot must be open-source. You will build upon previous work. See


Super Rapid Annotator - Advanced Multimodal Video Annotation Agent

Mentor: Raúl Sánchez Sánchez (raul@um.es), Manish Kumar Thota (manish.thota1999@gmail.com), Cristobal Pagán Cánovas (cpcanovas@um.es), Rosa Illán Castillo (mariarosario.illan@um.es) and team. 

Default but negotiable size: medium 175 hour project. 

Difficulty: MEDIUM

Objective

Building upon the foundation of the previous Super Rapid Annotator project (https://github.com/manishkumart/Super-Rapid-Annotator-Multimodal-Annotation-Tool), this initiative aims to develop an advanced annotation agent that leverages state-of-the-art multimodal large language models (MLLMs) and reasoning models. The agent will process videos and generate structured CSV outputs for annotation purposes, operable via a command-line interface (CLI) or Python.


We have a software called Rapid Annotator(https://sites.google.com/case.edu/techne-public-site/red-hen-rapid-annotator). Students upload a bunch of videos and watch them one by one annotating if the person is inside or outside, if it wear glasses, … We want to automate when possible it and avoid the repetitive tasks to the students and get a resultant csv with all the annotations using a multimodal model.

System Components

Agent-Based Annotation System:

Command-Line Interface (CLI):


Please, visit https://raul-sanchez.notion.site/super-rapid-annotator-2025 to get more info and a more detailed explanation


Red Hen TV News Multilingual Chat - LLM

Mentor: Tarun Jain, Sridhar Vanga and Karan Singla (tarunjain@gmail.com, sridharvanga2001@gmail.com, ksingla@whissle.ai). 

Default but negotiable size: medium 175 hour project. 

Difficulty: MEDIUM-HARD (we will only consider exceptional proposals on this)

Objective

In summer 2024, We created dataset based on 1 year of T.V News using broadcast recordings from news channels in 5 different languages. Red Hen boasts access to a large news archieve, processed with speeech and natural lanuage processing pipelines over previous google-summer-of-code, and on-going improvements to Multi-modal AI pipelines and collaborative efforts.


In the scope of this project:

-- We extend our LLM capabilities to more languages and larger data.

-- Create data-creation-tool to store vector DB's with metadta and generate Question-Answering pairs to create an updated version of model.

-- Setup inference tool to use RAG, and LLM efficiently with an interface to report verifiable results for Broadcast News.

Baseline project

You can see model and datasets from previous year at: https://huggingface.co/RedHenLabs

Released pre-print: https://arxiv.org/pdf/2410.07520

Github repo: https://github.com/RedHenLab/TV-News-Chat-LLM

Skills required:

Proven experience with fine-tuning open-source LLMs. Create efficient pipelines and experience with creating chatgpt kinda interface for Red Hen News LLM. 

Experience with fine-tuning LLMs,  bash and python scripting, React-based front-end and back-end creation will be ideal.

Multi-modal T.V. News Real-time Transcriber

Mentor: Karan Singla (ksingla@whissle.ai), Sridhar Vanga

Difficulty: MEDIUM, 

Duration: Medium


Objective

 Voicebased technology has recently seen democraticization methods to deploy.  We want to provide a speech recognizer, which can perform transcription, voice-bio and NER using a model trained on large-scale broadcast news.

In the scope of this project, we will provide pre-trained models and also provide a recipe to host it on your machines for useful applications.


we will update more details soon.


This will be a news-focused work on our research efforts on audio-visual 1-step voice understanding.


Relevant Reads

https://kolubex.github.io/projects/gsoc_2024/

https://aclanthology.org/2023.icon-1.29/

Skills required:

Proven experience with processing large collection of audio for speech recognition.

Experience and understanding of fine-tuning E2E ASR systems (for e.g: Conformer / Citrinet / Wav2Vec2 / EspNet models)


Modeling Wayfinding

Possible mentor: Mark Turner (turner@case.edu) and Francis Steen (profsteen@gmail.com),  

Default but negotiable size: medium 175 hour project.

Difficulty: MEDIUM-HARD

Description: Develop a mathematical and computational model of human decision-making using the Wayfinding theory, a process where individuals navigate through a complex space of possible actions. Your project should model how individuals make decisions when faced with limited time and cognitive resources, leading to choices that are formally sub-optimal yet resource-rational.  For example, to develop a formal and computational model that captures these dynamics, you may begin by formalizing a "choice" functional with sub-functionals representing priorities. Each priority sub-functional can then be weighted by an evolving activation function that activates a subset of priority sub-functionals at each timestep to simulate changing priorities.  

The task is designed to develop a general model of decision-making as an alternative to game-theoretic models. For background see

McCubbins, M. D., & Turner, M. (2020). Collective action in the wild. The Extended Theory of Cognitive Creativity: Interdisciplinary Approaches to Performativity, 89-102

McCubbins, C. H., McCubbins, M. D., & Turner, M. B. (2018). Building a new rationality from the new cognitive neuroscience. Handbook on Bounded Rationality, Routledge Publishing House (Forthcoming), Duke Law School Public Law & Legal Theory Series, (2018-52). 

The application domain can be various scenarios such as market behavior, communicative interactions, or animal foraging.  For instance, you could model the movements and foraging behavior of a unicellular organism such as a paramecium. The organism is in an environment where food is unevenly distributed and the organism must expend energy to move. Let's grant it some simple sensory capability such as smell, some ability to learn and remember, some ability to discriminate between candidate nutrients and selectively ingest, and some ability to monitor its own energy reserves. Each decision incurs a cost -- the cost of sensing, of comparing with a past sensory datum, of generating a strategy, and of carrying out that strategy. In any given situation, the paramecium needs to decide how much time and energy it allocates to sensing its environment to identify potential food sources, generating a dimensional map of options, assessing the different possible ways forward, moving and feeding, and during this process tracking its energy levels. Your model should visualize how the decisions of the organism vary depending on the cost of the various tasks, such as sensing, assessing likely costs relative to benefits, moving, and feeding. 

We're just spelling out one example of a domain you could use to explore the dynamics of wayfinding. A critical feature of wayfinding is that information-processing carries a metabolic cost and needs to generate a payoff. You want to build the model so that it can be elaborated -- for instance, the goal dimension could be expanded to introduce reproductive opportunities; the strategy to sense and move could be expanded with an energy-conserving state if a minimal threshold is unmet; the sensing could be expanded to include the threat of predation, and so forth. The model should allow us to explore the tradeoffs between cognition and action. In the paramecium example, we would for instance be interested in exploring the minimum information-harvesting and processing requirements for survival and to visualize the consequences of varying the cost of the various cognitive and motoric processes.

We have spelled out one example of a conceptual model but you are invited to develop others. We encourage you to develop a simple model system that brings out decision-making dynamics in situations where an agent's priorities varies between different goals, where there are multiple possible solutions that differ in their cost and payoffs, and where these costs and payoffs are associated with various degrees of uncertainty. A key part of the wayfinding model is that both information processing and action incur a cost, creating a dynamic tradeoff between sensing and assessing on the one hand and acting on the other. The measure of cost should include some proxy of energy and time, but could also have additional dimensions.

Expected outcome: A working model in python or C++ 

Skills required: Some background in computational modeling and mathematics. Preferred platform Google Colaboratory

Quantum Simulator

Possible mentor: Francis Steen (profsteen@gmail.com) and Mark Turner (turner@case.edu)

Default but negotiable size: medium 175 hour project.

Difficulty: MEDIUM-HARD

Description: A range of social processes cannot be accurately modeled by classical machine learning.  Variance in behavior is often high; even the same individual may respond differently to the same situation. In addition, people innovate and generate new behaviors both in competitive and in cooperative interactions, 

Quantum simulators allow researchers to test quantum error correction codes and experiment with novel quantum architectures. In recent years researchers have developed and tested quantum variations of a number of popular social algorithms using classical quantum simulators; highlighting their utility and the advantages of creating widely accessible quantum simulators.

See for instance Li, A., & Yong, X. (2014). Entanglement guarantees emergence of cooperation in quantum prisoner's dilemma games on networks. Scientific reports, 4(1), 6286.

The task is to select an existing social algorithm or to propose a novel quantifiable social interaction and build a quantum simulator for it.

Expected outcome: A working model in python or C++ 

Skills required: Some background in quantum circuits and quantum statistics.

Quantum Wave Function for Information-Processing

Possible mentors: Paavo Pylkkänen (paavo.pylkkanen@helsinki.fi), Francis Steen (profsteen@gmail.com), and colleagues

Description: Neural networks are descendants of McCulloch & Pitts' threshold-based mathematical model of a binary neuron, but there is ample evidence that unicellular organisms are capable of relatively complex maze navigation and other cognitive tasks, indicating information-processing capabilities in cellular subsystems. For example, Picard & Shirihai (2022) argue that "mitochondria are the processor of the cell, and together with the nucleus and other organelles they constitute the mitochondrial information processing system." Light-harvesting complexes in chloroplasts  have been shown to rely on quantum tunneling; cells may also have recruited quantum physics for other functions, including information harvesting.  According to Bennett & Onyango (2021), "Mitochondria can be viewed as transitional organelles that bridge the quantum world of very small wave-particle behavior and the classical world of decoherent larger, more macroscopic structures such as cells."

The task is to develop a computational model of elementary processing of analog information using Schrödinger's wave equation, leveraging the fact that a quantum wave function has multiple valid solutions and only one of them manifests.  Heisenberg characterized the elementary particles of quantum physics as "not as real" as things or facts; instead "they form a world of potentialities or possibilities", suggesting a potential use in information processing.

According to the de Broglie-Bohm Pilot Wave theory, the particle is associated with a quantum wave that interacts with its environment; for instance, in the double-slit experiment, it passes through both slits and interferes with itself.  The particle  responds with its own energy to the  form of this wave, similarly to how a model airplane responds to a guiding radio wave.  

In Bohm & Hiley's elaboration of Pilot Wave theory, the Quantum Hamilton-Jacobi Equation can be decomposed into a classical component and a quantum component, the quantum potential. Most of the energy is in the classical component, but a small part of it is in the quantum potential. As the energy in the quantum potential is informed by and responds to the shape of the quantum wave, the process creates new information, expressed in the trajectory of the particle. The diagram shows the Bohmian particle trajectories.

Simple unicellular organisms may have recruited this information-generating process for survival purposes. For instance, it could be used to categorize an analog sensory signal and serve to trigger an appropriate motor response.  The task is to develop a mathematical and computational model of this process. 

An approach to quantum decision-making is described in Engel et al. (2007). Evidence for wavelike energy transfer through quantum coherence in photosynthetic systems. Nature, 446(7137), 782-786), available on request. They write,

"[S]uperposition states formed during a fast excitation event allow the excitation to reversibly sample relaxation rates from all component exciton states, thereby efficiently directing the energy transfer to find the most effective sink for the excitation energy (which, in the isolated FMO complex, is the lowest energy state). When viewed in this way, the system is essentially performing a single quantum computation, sensing many states simultaneously and selecting the correct answer, as indicated by the efficiency of the energy transfer."

In this account, the excitation event (in effect, the particle) is given agency: the superposition states enable it to "reversibly sample" multiple possible trajectories and realize the optimal one every time.   This presents a highly efficient paradigm for quantum computation; however,  it makes claims that are incompatible with the widely accepted Copenhagen Interpretation.

In Google Summer of Code 2024, Red Hen Lab mentored two projects that studied decision-making in the unicellular organism Stentor roeselii, focusing on a small number of identifiable avoidance strategies (for background, see Dexter, J. P., Prabakaran, S., & Gunawardena, J. (2019). A complex hierarchy of avoidance behaviors in a single-cell eukaryote. Current biology, 29(24), 4323-4329). A task for 2025 is to model in detail how these strategies are actually implemented.

Stentor roeselii has hundreds of cilia that are involved in carrying out the high-level avoidance strategies. Each cilium is constructed out of a small number of microtubules with a rotor at the base. In order to effect a strategy change, a considerable number of rotors must move in an orchestrated manner. The task is to model this process of orderly responses. 

The purpose of the model is to help us understand how a single-celled organism -- that is to say, an organism without a brain, without neurons -- is capable of complex high-level decision-making. Are the high-level strategies described in Dexter (2019) the result of local decision-making at the level of the individual cilia or evidence of a hierarchical decision-making system? If the latter, what is the minimal amount of information that must pass between the levels in either direction?

Red Hen is open to a range of approaches as paradigms of biological information processing using  the quantum wave function.

Expected outcome: A working computational toy model of quantum information processing that can serve a platform for iterative improvements and elaborations

Skills required: A basic familiarity with Schrödinger's wave equation and preferably some experience with computational modeling of dynamic systems

Wayfinding in Artificial Life

Mentors: Francis Steen (profsteen@gmail.com) and colleagues

Difficulty: MEDIUM

The Neo-Darwinian synthesis holds that biological evolution proceeds through a process of random variation and natural selection.  In this project, we invite proposals for modeling a wayfinding component in evolution, using an artificial life simulator.

A simple proposal that a candidate may explore is that living organisms are capable of accumulating surplus resources, including surplus energy.  Friedrich von Schiller proposed surplus energy as an enabling condition for play, a widespread mode of animal behavior with deep evolutionary roots. The task of modeling a wayfinding component in biological evolution can be addressed by developing a mathematical approach to play behavior as a form of wayfinding. The aim of the model is to explore the logic of using play behavior as a means to explore possibility space in a strategic manner to identify potentially fruitful new forms of behavior. These new behavioral patterns can then be subject to natural selection in an ongoing dynamic process.

Skills required: Experience with artificial life platforms.

TV Recording and Processing Pipeline with Apache NiFi 

Mentors: Peter Uhrig (peter.uhrig@fau.de), Armine Garibyan (armine.garibyan@fau.de), Raúl Sánchez Sánchez (raul@um.es), Cristobal Pagán Cánovas (cpcanovas@um.es); default but negotiable size: medium 175 hour project.

Difficulty: MEDIUM

Red Hen is upgrading its European TV recording facilities at FAU Erlangen-Nürnberg and at the University of Murcia. For the recording, quality control, recoding,  and automatic annotation of these broadcasts, a robust and flexible pipeline using Apache NiFi should be designed. It should interface well with various Linux servers and with High Performance Computing systems running SLURM. Compatiblity with Red Hen's YouTube pipeline is will also be a necessary feature.

Required skills: Fluency both in Bash scripting and a good understanding of pipelines, including definitions of interfaces and data structures for interoperability. Knowledge of text-based data formats as well as audio and video codecs. 

Detection of Intonational Units v2

Mentor: Peter Uhrig (peter.uhrig@fau.de);  default but negotiable size: medium 175 hour project.

Difficulty: MEDIUM-HARD

This project attempts to replicate the AuToBI system with modern machine learning applications (See Andrew Rosenberg’s PhD thesis for details). It builds on top of last year's GSoC project by Prakriti Shetty. 


Required skills: Strong machine learning skills and experience with audio processing. The methods used in AuToBi were state-of-the-art more than 15 years ago. With the advent of large pre-trained models, we expect to be able to improve on that baseline. You need a good understanding of annotation, the ability to work with obscure file formats and to extract relevant information from them, i.e. good data processing skills.


An LLM Appropriately Responsive to Different Audiences

Mentor: Francis Steen (profsteen@gmail.com), Leah Luong <communications@kfa.org>,  Francisco Mazza <francisco@kfa.org>

Default but negotiable size: medium 175 hour project.

Difficulty: MEDIUM

A key challenge for LLMs is targeted answers. When human beings communicate, they adjust the content to fit what they know about the recipient and the present circumstances. LLMs lack this ability. Jiddu Krishnamurti (1895-1986) was a public speaker over a period of some 70 years addressing a large variety of audiences. He addressed children very differently from adult public audiences, and he spoke to people in different countries and continents in ways that took into consideration the local culture.The current task is to train an LLM based on Krishnamurti's recorded speeches and conversations that will modify its responses based on the profile of the questioner, starting with age and geographical location.

We suggest using NotebookLLM for this task, but are open to other proposals. The task requires a system where users create accounts that provide some information about them, such as age and location, which are then used to alter the responses given by the LLM. The proposal should contain a detailed plan for how the metadata available from Krishnamurti's talks and discussions will be utilized to alter the responses to questions as a function of parameters in the user's profile.

Required background: Experience with LLMs

Intelligent Information Retrieval using Multimodal Inferences

Mentor: Sabyasachi Ghosal (saby.ghosal@gmail.com );  default but negotiable size: medium 175 hour project.

Difficulty: MEDIUM-HARD

Red Hen has access to a large news archive. Over the years in google summer of code many pipelines have also been developed in the domain of Multi-modal Communication. This project will involve a user asking questions about TV news videos using voice queries and receive intelligent information using spoken and visual summaries. For the inference multiple input and output modes like the audio, hand gestures, emotions (face & audio) etc. can be used. This project will only focus on English language.  

The story line can be something like this, where user interacting with the system will ask question like this.