News Recommendations
News recommendation algorithms influence far more than what stories we click—they can shape our understanding of the world. In this episode, Kyle Polich speaks with Andreea Iana about responsible AI, filter bubbles, multilingual news recommendation, and her open-source NewsRecLib framework for evaluating recommender systems. They explore why bigger models aren't always better and how future recommendation systems can balance personalization with diversity and societal impact.
Guest
Andreea Iana: I am a postdoctoral researcher in the Data and Web Science Group at the University of Mannheim. I hold a PhD in Recommender Systems from the University of Mannheim, where I was advised advised by Prof. Heiko Paulheim (University of Mannheim) and Prof. Goran Glavaš (University of Würzburg). During my PhD, I was a visting researcher in the WüNLP Group at the University of Würzburg. Before that, I completed my M.Sc. in Business Informatics at the University of Mannheim, and my B.Sc. in Liberal Arts and Sciences from the University College Maastricht, in the Netherlands. I work on advancing responsible and inclusive AI for information access, with a particular focus on multilinguality, cross-lingual IR, and investigating and mitigating algorithmic bias in retrieval and recommendation systems.
Transcript
Kyle Polich : Welcome to Data Skeptic a podcast exploring the methods use cases and consequences of recommender systems
Kyle Polich : Welcome to another installment of Data skeptic Recomer Systems Today on the show we take on the topic of news recommendations Now at a glance this could seem like the type of problem where you could quickly adapt existing techniques to use Why can't you recommend news with the same methodology as you use for recommending movies
Kyle Polich : Well there's a couple of reasons for that Obviously movies can be more evergreen whereas news stories are pretty timely but also you get different feedback I'm happy to give thumbs up to a movie I like I'm not going to thumbs up an article that's well written about how innocent
Kyle Polich : And people died Those differences aside the stakes are pretty high here If you recommend the wrong movie at worst you've wasted a person's time If you consistently recommend the wrong news stories you could introduce bias or put a person inside of a bubble
Kyle Polich : You know the information highway has brought us a deluge of information more than any one person could consume in a day So news recommendation systems are becoming more important than ever and we'll be exploring that topic in today's interview
Andreea Iana : Hello my name is Andrea Ayana I'm a postdoctoral researcher in the Data and Web Science Group of the University of Mannheim I'm interested in working on responsible and inclusive AI for information access and I have uh recently finished my PhD where I focused on neural news recommendation
Kyle Polich : Very cool Um before we get into news recommendations could you expand could you expand a little bit on responsible and inclusive AI What does that mean to you
Andreea Iana : So let me maybe start with the news recommendation domain where I focused on on this topic So here I try to look at the problem uh from 3 different angles to take a step back uh when most people think of recommender systems they think about um let's say uh products or music or movies uh but a very important domain is actually the news domain because nowadays a lot of the platform
Andreea Iana : Platforms that we are using to access information have some recommender system running in the background that filters the information that we are trying to access and personalizes it to our taste So in a way recommender systems influence the information that we are exposed to and basically they can shape our view of the world because of the information that we can actually get access to
Andreea Iana : And especially this is very important since news are a pillar of democratic societies and by filtering some of the information that is not deemed relevant to our tastes or our preferences we might not be exposed to all sides of the debate for example and this has been shown that it can create so-called filter bubbles So basically an environment where people are exposed only to information that agrees to their own opinions or to their own
Andreea Iana : Views of the world and this can in the long term lead to things like opinion polarization or radicalization So in this area what I understand by a responsible and inclusive AI is basically AI or systems that do not only care about performance for example increasing click-through rate on a website or on a certain news but also systems that care more about the broader societal values that
Andreea Iana : as diversity of opinions inclusion of different users for example something that I focused a lot in my PhD was studying whether different biases occur in recommender systems or whether recommender systems tend to for example prefer certain types of viewpoints or sentiments of news articles whether these follow for example user preferences or even amplify this
Andreea Iana : And how we can actually try to mitigate these effects and also how we can be more inclusive in terms of linguistic inclusivity because um for example I speak multiple languages and a lot of people in the world are polyglots and they consume um news in multiple languages and some of these languages are not so high resourced so they don't have a very high digital footprint like English
Andreea Iana : And we've seen over the years that systems don't perform so well in languages that are not English or that are further away linguistically from English So how can we actually create a system that would also support speakers that speak less represented languages or that want to consume news in parallel in multiple languages
Kyle Polich : Do you think it could be the case that uh AI will just solve that for us it does seem to be pretty good at languages I agree I maybe English primarily since that's most of the training data but uh do we just need more time and it'll learn all the languages or do you think special care is in order
Andreea Iana : I mean over the years it got a lot better at other languages but I would say we're still quite far away from
Andreea Iana : Uh Representing all the languages and right now existing models so everyone talks about large language models only support a couple of tens or maybe hundreds of languages but we have over 7000 languages in the world and some of these are not even represented via text so they are mostly spoken which means that we don't even really have training data to include them to to train AI models for them
Andreea Iana : So I think we're still quite far away from that Of course we are closing the gap but only for certain languages And as I've seen in my own work if you target a language that is quite dissimilar from English or some other Western languages the performance of any model drops tremendously
Kyle Polich : Did you take an interest in recommender systems and then find news as a you know particularly novel outcome or were you interested in news and found your way to recommender systems
Andreea Iana : To be honest I started with recommender systems and this is a bit of an older story So I had my first encounter with recommender systems at the end of my bachelor where I worked a bit on a recommender system for um basically recommending courses to students based on which courses they've already taken and how they want to build the curriculum And then later on during the masters
Andreea Iana : I went a bit more into scientific recommender systems So for example where should I publish a paper that I've just written at which venue which academic conference is most relevant for me And then when I decided to do a PhD it was a mixture of everything So I had this hate love relationship with recommender systems I liked them but I didn't like that they narrow our perspective
Andreea Iana : On what we're seeing everywhere And um there was a project that was looking exactly at responsible recommender systems in the news domain So this is how I came across this and I got very interesting in this because the news domain itself is quite special So it shares a lot of similarities with again a product or music recommendation but what makes it special is that it's very societally relevant and not only that but also the characteristics of the news that we're seeing So
Andreea Iana : We have thousands of news published every day which whose relevance decays extremely rapidly Though while a movie can still be relevant in 20 years a lot of the news that we see today might not be even important tomorrow or in a few days as we have this extremely high churn of news Also for example user feedback is not explicit so we only know that OK the user clicked on a particular
Andreea Iana : Article but what does this actually say Was the user just interested or uh in that article or just clicked on it and then closed it and so on and so forth So that's how I started on this
Kyle Polich : You hit on a very interesting point there uh when it comes to product reviews music books movies whatever we're thinking of rating it 5 stars or thumbs up thumbs down I don't think I've ever rated a news article How do you get
Kyle Polich : feedback
Andreea Iana : Yeah so as I've mentioned feedback is mostly implicit so what we take as a proxy for the user feedback is usually clicks and then there's a couple of other signals that there can be for example the time spent on an article
Andreea Iana : If it's very short a couple of seconds you probably can consider that the user clicked on it but was not interested
Andreea Iana : Yeah that's mostly implicit feedback in terms of clicks uh the time spent from the user side I would say because um there's also a lot of maybe demographic information that you would have in other for other domains that you don't have uh for the news system And again for the for the news domain um a lot of these interactions are limited to a single session because at least when we work with academic data sets we know that
Andreea Iana : So maybe to take a step back here since the user is not logged in in a lot of the these uh platforms we cannot track um the user's clicks across um very long sessions So we have very short time spans in which we know OK the user clicked on these 30 news compared to for example movie recommendation where we can um follow the user on a certain platform across the years and we have a very long uh user history from which we can learn the preferences
Kyle Polich : Well I have some questions I'd like to deep dive into on news rec lib but I guess before we go too deep could you just define what this library is What is the project you worked on uh what is its function
Andreea Iana : Yeah sure So in the beginning of my work in news recommendation a problem I really encountered was that uh we have a lot of different systems that are implemented in different ways on different platforms using different uh libraries and capabilities and I found it quite hard to compare meaningfully and to to compare apples to apples sometimes because we look at the paper we see some numbers reported from some models that claim to be better than other recommended systems
Andreea Iana : And there's a lot of moving parts One problem is that everyone implements their own system and what I try to do in news newsrecli is um put all the system in one place and in a way such that we can uh keep uh we can decouple most of the building blocks of the recommender system and keep whatever we want to keep fixed so that we can see
Andreea Iana : Which parts of the model have the highest impact on performance and so on and so forth So for example what you can do in newsrecli is combine different models from different models So let's say we take the part that is responsible for uh creating a representation of the news from one model and we couple it together
Andreea Iana : With the part that uh creates sort of representation of the user profile in another model and we see we can just mix and match everything And the idea is to create a platform where we can have a very rigorous evaluation of different models on different data sets and everything is uh highly reproducible
Kyle Polich : Well there's a a common example of e-commerce where uh historically I guess you'd have handcrafted features anymore it's probably an embedding of some kind you embed the product and you embed the user too based on what you know about their preferences Is this a pretty direct uh uh analog or have you had to model the problem of news in a different way
Andreea Iana : I would say it's quite similar apart from the since I looked at neural news recommendation we don't use handcrafted features but what we do is let a neural network create these representations of news and users So to briefly say how a neural news recommender looks like it has a couple of building blocks One of them is a news encoder that takes an article for example the title of an article or the headline or maybe some other feature
Andreea Iana : Like the category or the viewpoint and creates a representation of that article a numerical representation that can be used by the the model Afterwards And then we have the uh user encoder that aggregates the representations of the news that the user has clicked on into a user representation which is meant to capture preferences of the user um what based on what they have clicked on Again
Andreea Iana : We take that as a proxy for what we think the user is interested in And then for each candidate article that we have in our large news pool we try to see if it would be relevant for the user based on this um user profile So from that point of view I would say yes this matches And the idea is OK um how do we construct the user encoder How do we construct the news encoder
Andreea Iana : And then based on how we select these different uh building blocks uh how we can create the overall architecture So Newsreclib has some predefined models that were state of the art at the time uh when I created it But of course any new model can be added to the library for example just by specifying a new type of user uh news encoder or user encoder or you can as I mentioned already take uh different building blocks that are already implementing that and just mix and match them
Kyle Polich : So would you describe News rec Lib as like a framework in that regard that I could bring my own model
Andreea Iana : Yes exactly something like that And then you can just as I said um yeah build your own building blocks there and then just test it on existing data sets or if you have a new data set for example also included in the framework such that new models from other developers could be tested on it
Kyle Polich : Are there common data sets that you would recommend to listeners if they were embarking on a similar
Kyle Polich : path
Andreea Iana : Uh yeah sure So one of the very used data sets was the Microsoft News Recomer data set but this has now got quite a bit old A newer data set that has been published I think 2 years ago it's um Ebner from a Danish news media company
Andreea Iana : And there's a couple of other smaller ones but in general one big problem I guess for the news domain um is that um a lot of these data sets so the the media companies that have this data don't really uh make uh open sources data sets So we are working with just a couple of data sets that over time of course also get um a bit outdated and also very highly um differing
Andreea Iana : The sort of articles they actually contain So for example the mind data set has a lot of what we call soft news a lot of entertainment news or sports uh while Ebner has a lot more of hard news that contain more politics or more sensitive topics
Kyle Polich : What was your original motivation for starting the news reclib project
Andreea Iana : In a nutshell I had to work with a lot of different models and yeah I got a bit frustrated at the lack of comparability and how hard it is to get everything to work So I thought OK it's easier to spend all the time to put everything in one place and then it's a lot easier
Andreea Iana : To just incrementally make new changes or bring in a new model um and this is especially useful when you want to have reproducible um evaluations and reproducible work because everything is um built on the same let's say um underlying framework Well
Kyle Polich : there's definitely a perspective in this day and age that maybe bigger is better when it comes to models Let me take the one that has the most parameters and I'll just assume that's the best for me What's your perspective on it
Andreea Iana : Yeah I really think it depends on what we're looking at and um as I've seen in my work this is not always the case So I also started with this assumption that more complex architecture actually bring in more performance
Andreea Iana : And better performance but one thing that I came across during my work and then continued to investigate that was quite surprising is that um as I said this is not always the case So what we've discovered is that sometimes simpler is better and here I refer mostly to the user side
Andreea Iana : So in our peeling back the layers what we actually tried to do was to decompose the building blocks of the recommender system and try to understand what exactly drives performance and this is because over the years uh every new system that was being proposed was more complex or introduced some tiny change compared to an existing model and they reported maybe a couple of percentages improvement or even less than a percentage improvement over existing systems
Andreea Iana : So it was very hard to judge OK do this more complex or incremental changes actually bring anything meaningful And what we try to do is um first of all decompose as much as possible these different components and using code or the user encode or how we combine everything
Andreea Iana : And try to keep fixed as many parts as possible and just move on at a time to see OK what is the impact of this part on the performance of the system And here we didn't look only at performance in terms of what is usually measured in offline evaluation So for example how good is the model at ranking the relevant candidate articles but we also wanted to understand hey do these two different models actually
Andreea Iana : Recommend the same thing or not and do they actually create very similar representations of news and users or not And what we actually found is that what really drives performance in terms of yeah ranking similarity and everything in in our setup is the news encoder So what matters very much is how good are the representations of the articles that we are creating
Andreea Iana : And here of course I would say maybe a bit the bigger is better because indeed if we use um for example pre-trained language models or language models that were trained on vast amount of data usually sourced from the internet already know how to capture a lot of intrinsic knowledge from the news articles how to to capture for example topics
Andreea Iana : From the text itself So here yes bigger is better but uh at the same time we've noticed that on the user modeling part bigger is not better So um there we actually had a very surprising result that more or less just taking the representation of the the articles that the user clicked and just averaging these representations performs um equally better to a lot more complex
Andreea Iana : Approaches and why we think that's the case is because as I've mentioned before news recommendation is very different than other domains where we have very long histories So on the one hand on such short histories maybe we cannot capture some sequential signals that we would capture for example in e-commerce or music or movies and on the other side again such powerful
Andreea Iana : Language models already capture all the the needed knowledge from the text itself So here at least in my work I mostly uh looked at content-based news recommenders So this means that we looked at systems that we cared mostly about the content of the articles and this is text And again this is different to maybe music recommendation where um you care also about the audio part of um of a song
Andreea Iana : By keeping this all these parts fixed and then just varying for example we keep the the news encoder fixed and vary the different user encoder approaches that we've seen out there we saw that OK what what really drives performance is the underlying news encoder and again not only in terms of ranking performance but we've what we found quite surprising is
Andreea Iana : Also in terms of the similarity of the recommendations that they produce So for example if we take a list of recommendations of 10 we found out that in 70% of the cases no matter how different the user encoders are we have the same set of recommendations So in practice we maybe see a difference of 0.5 improvement in ranking
Andreea Iana : But what does this actually mean when we look at what's recommended and there's not much difference And yeah I think this is very important in practice because what it tells us is that we don't need to invest time so much into making incrementally more complex changes but maybe taking what's out there understanding what really drives performance and then using some simpler approaches
Kyle Polich : I think I'm correct in characterizing it then saying that uh the news encoder is where you found the most performant qualities and I wanna drill down there but before we jump into it uh when you think about the user encoding
Kyle Polich : Do you think the reason that there's no more uh juice for the squeeze is because the methodologies are you know perfectly state of the art and great or is there something lacking maybe in how uh the solicitation of feedback that it's just not a rich enough environment What's your perspective on why the user encoding is kind of maxed out
Andreea Iana : Mostly the second one I think so because in the first one I'm not sure if we we showed that just taking the average of the representations of the news that the user consumed is performing on par with very complex approaches so maybe it's not that we already have the state of the art there I think it mostly comes from the signals that we have from the user
Andreea Iana : And this comes a lot to the data that we have In particular again we only have clicks and we assume that the few clicks that we have that the articles that the user clicked on mean that the person is interested in that article And we know this can also be for example a lot of clickbait or sometimes just randomly clicking on an article
Andreea Iana : And closing it especially that a big problem with datasets in academia is that we don't have signals such as dwell time all the time So we don't all the time have information about the dwell time so how much time did the user actually spend on certain article to actually say OK they clicked and spend time on it so then it must have been really important to them
Andreea Iana : So I think generally yes having richer signals and richer data would would really go a long way to actually seeing where this user modeling helps because um I believe that user modeling or user encoders are are still important but with the data that we have and with the the way we're modeling the news also we can't go much further than this so we would actually need richer data sets and more up to-date data sets
Kyle Polich : Well I hope somehow they exist one day that'd be a nice thing to come into being but in the meantime if we focus then on the news encoding if chasing complexity doesn't necessarily pay off bigger is not always better do you have any insights as to what does make one news encoding better than another
Andreea Iana : So a lot of the news neural news recommenders in the beginning used what was called a pre-trained word embedding So again before the large language models there were the word embeddings that's creating a static representation of some words and there are a lot of problems there because for example if you would have two words classic example the the bank where we get the money from or the bank of the river or the bank in the park would get the same representation
Andreea Iana : So the context was not captured by this and what was happening in the first iteration of neural news recommenders is that we were adding some neural networks on top of this pre-trained word embeddings to try to contextualize the meaning of this
Andreea Iana : And here we've seen that for example just looking at the text of an article produced decent representations of the news but in that case adding more information for example about the category of an article or some of the entities that were contained in the article So this means names of organizations or people or locations added
Andreea Iana : A lot more to the quality of the embeddings Of course when the language models got a lot better so they got to be contextualized during pre-training trained on the whole web corporate and so on and so forth adding these extra signals the benefits you would get from adding these extra signals from categories or from entities started to decrease and we've seen that
Andreea Iana : Yeah here the bigger the better in terms of the language model but even here for example we we can get very good representations with some smaller language models so we don't need to go with the latest embeddings that the biggest LLM can produce nowadays
Kyle Polich : Well when comparing two different models you know hopefully on the same data set it all comes down to some benchmark Do you think as a community we have the right benchmarks today
Andreea Iana : I would say no And again this is because most of the open source benchmarks that are out there so the ones publicly available are a few years old So even though Ebnerrd is 2 years old I think by now and yeah as I said the news domain is fast moving and changing and keeping up to date with that would be very important
Andreea Iana : And not only that but most of these benchmarks are focused only on the textual content of news and I think this is not the only thing that matters especially in today's world Images so multimodal inputs such as images or videos play a very big role because sometimes maybe people are triggered to click on a certain article because of the image that is attached to that article and not just because of the title A lot of news consumption has now changed A lot of people consume
Andreea Iana : News also via short videos or other means so not only textual news articles and these are not captured yet in the existing benchmarks that we have So as a community I would think maybe higher or a bigger collaboration between industry that has
Andreea Iana : Access to this sort of data and academia that works on a lot of methods on how to improve news recommendation and not only in terms of performance but again if we go back to other things like responsibility and inclusivity this would go a long way to to have fresher more up to-date and richer datasets
Kyle Polich : Would you mind commenting a bit on your thoughts on filter bubbles and the role recommender system plays in promoting them or getting rid of them or uh how do recommender systems cozy up to the problem of filter bubbles
Andreea Iana : Yeah sure So in a nutshell what happens with uh recommender systems is that they are usually optimized to to maximize the user engagement with something for example for click-through rates so we want the user to click more on certain articles and to achieve this engagement what they are usually trying to do is to maximize the relevance or the similarity
Andreea Iana : Of what's recommended to what the user has already consumed because then we know for sure OK this article will be interesting to the user And this is what usually creates this feedback loop in which the user consumes some news let's say some right-wing politics news and then the recommender knows OK the user likes right-wing politics so I will just keep recommending right-wing politics
Andreea Iana : And the problem is then that the user doesn't really see articles from a different viewpoint or perspective or maybe from other categories And here the evidence it's a bit mixed whether these filter bubbles are real or not depending on the studies that have been done but
Andreea Iana : We've also noticed in our own work is that recommenders do have a tendency to recommend slightly more negatively framed or negative viewpoints on articles that go beyond what's already in the data So we already know that a lot of for example politics news have a negative sentiment but what we've observed is that the news recommenders tend to still recommend even more negatively framed articles
Andreea Iana : So the problem is that then you're only exposed to what you agree with and this really can generally influence a person's perspective uh viewpoint and opinion in general
Andreea Iana : So this is why I'm saying that not only performance matters especially in the news domain but also taking into account these effects of recommender systems the societal effects and what we should strive for is balancing accuracy and performance in general with more balance in what we recommend more diversity and so on
Kyle Polich : Curious if you have any thoughts on how to optimize that it does seem that diversity of viewpoints is a good thing but like somewhere someone probably assumes the Artemis mission was filmed on a soundstage and the moon landing was faked and I don't think we need to necessarily promote that diversity of opinion you know how do we optimize for this
Andreea Iana : And I I would say that diversity should be real diversity or real balance when we we when we try to achieve it and not just pretend so we shouldn't recommend let's say articles from uh all the sides of the scientific or political spectrum for the sake of saying that something is balanced but it should also be recommended proportionally to the evidence So of course if um something is just a conspiracy theory we shouldn't promote it as highly as we would promote something
Andreea Iana : That is scientifically backed Uh how to achieve this Yeah that's a more complex question So on the one hand we should not only optimize for accuracy which is what happens in a lot of systems now and then optimizing for something like perspective diversity or um category diversity is already um a goal that it's optimizing for accuracy and optimizing for diversity are conflicting goals So we we have to find a balance between that
Andreea Iana : And then how to find the right balance I think it should be um not only something done by the system but it should be something that takes into account multiple stakeholders here because we have the writers of the articles we have the editors we have the the users themselves we have maybe the more um societal-related goals as a society we do not want to promote fake news for example
Andreea Iana : We don't want to promote I don't know conspiracy theories or also not sensational news Those would maximize for example the click-through rate but it's not something that we would like users to always click on So in general I think to achieve a true balance we should have multi-stakeholder recommender systems that try to balance the interests of all the stakeholders
Andreea Iana : which again goes into different and even more complex problem how to balance this not only perspectives but how to balance multiple stakeholders But in general yeah we should take a step back from just driving performance to to try to include a bit of diversity and how to do this I would say again should be a mixture of maybe let's call it some default values
Andreea Iana : So if we would have a system on a website that allows users to let's say turn the knob I want more diversity with respect to category or I want more diversity with respect to the perspectives that I'm seeing there should still be some hard coded or default values that don't allow people to go beyond
Andreea Iana : Something that is reasonable in a society So yeah again you you can say OK I want more diversity and I want to see this point of view from the left and the right wing side but it shouldn't allow people to just read the conspiracy theories for example
Kyle Polich : Yeah well when I go to one of these streaming video sites to find a TV show or a movie it's pretty clear to me they're doing some recommendation it's a little bit personalized to me especially if I've been there before I'm not certain if I'm encountering news recommendations in my daily life uh you know if I just go to a news website maybe everyone sees the same just like the old printed out days or whatnot where do the typical people encounter news recommendations in their life today
Andreea Iana : Yeah so I would say maybe if you go to the platform of a specific news magazine for example you maybe don't see it so much but especially on aggregator platforms so Google News for example if you go to Google News and you click on a couple of news and then you close it down and then you go back at least I keep seeing more or less news coming from the same category or topic So for example um
Andreea Iana : I attended a half marathon in a city and now I keep getting news about that city itself or half marathons around the world or so on and so forth So I would say uh in these sort of places and I would say it's also a lot of how people consume news nowadays So uh we have seen in the past that consumption has really changed from going to a specific website of one publisher
Andreea Iana : To go into these news aggregators and just um looking for a specific topic or a specific event that happened in the world and then getting news that are aggregated there from different publishers and this is really where a person uh recommendation and personalization comes in
Kyle Polich : A lot of your work has uh maybe I could characterize it as saying that there's a um complexity for its own sake is not the virtue
Kyle Polich : That probably at the direct fault of the machine learning community we've pushed our methodologies and that's worked in a lot of fields and brought us large language models and things like that but more is not always better I guess at least in news so what is better where would you like to see the field focusing its efforts
Andreea Iana : I would say on the one hand just like any field I think we should uh from time to time take a step back and have a really good look at what we're trying to achieve what we have achieved and really if we made any progress because as I've seen in my own work and as we've seen in the broader area of recommender systems also in other fields sometimes we push for a lot of complexity or newer
Andreea Iana : Models but then it turns out that comparing apples to apples in a fair setup actually older and simpler methodologies win But apart from that I would say yeah not caring just about performance and looking now OK the systems are already performing very well for English speakers for recommending exactly what you would like to see but how about other things So how about if I want to see
Andreea Iana : Actually not all the time the same thing What if I like a bit of serendipity and what if I like to see multiple perspectives on something or especially uh what it's very interesting What if I want to read news in multiple languages and here I don't want to read the same news in two languages Maybe I want to read about international news in English and then I want to read about local events
Andreea Iana : German since I live in Germany And how do we make systems that actually support this kind of thing such that I don't have to go to different platforms and search for these things myself and then sort of aggregate so I don't have to switch between platforms myself but I would just go to one place and I would get all this news in the languages that I want to speak And again I speak very well represented languages but what if someone wants to have access
Andreea Iana : Information and they only speak Georgian for example and we know that Georgian is is not so well represented so then do they actually have to turn to English to benefit from this personalized news recommendation or do they have to rely on machine translation which again is not very good because it doesn't perform so well in Georgian and so on and so forth So I think we should
Andreea Iana : Not only focus on performance but also on a lot of other things that are important for us as a society and also for different users such that not only Western centric content is well represented but that we can generally serve users across the world more equitably
Kyle Polich : Yeah I think if there was let's say uh something going on in another part of the world maybe a conflict and I wanted to understand it better
Kyle Polich : Ideally I would hear it from the actual affected people in their native words which is only an English speaker I can't quite do that so I guess I would go for translation in that case Do you think we'll get translation close enough at some point or will this always pose a challenge that uh there's no true translation
Andreea Iana : I think we're getting translation better and better but we're still not a perfect translation So nowadays I'm I'm working also on a data set for something not related to news and um working with different languages we can really really see how translation degrades when we go further away from English and especially if you have some terminology
Andreea Iana : Or domain specific terms especially if these don't maybe have a counterpart in English and a lot of things that are missed in machine translation is um generally maybe some expressions that are common just to a certain language and they they cannot be translated word by word or exactly in English
Andreea Iana : And this still works for for languages that uh as I said before have a written form but we have a lot of languages that don't have a standardized written form that are mostly spoken and their machine translation doesn't help very much uh at the moment So this is the where a lot of interesting work could be still be done and in terms of yeah machine translation language modeling and all these um technologies
Kyle Polich : Well I think if the news recommender system community were to hyper focus on let's say accuracy the classic metric of machine learning they're probably optimizing in a good direction but also to the wrong thing you know that doesn't fully represent our goals Do you think the goals of making news recommendation best for everyone or maybe you have some thoughts on what that even means but can we express it in a loss function Is it something algebraic or is there a uh something more to it
Andreea Iana : I don't think we can represent everything that we want in one loss function no So for example in one of the works that I did uh we were looking at how to yeah how to mitigate this problem of filter bubbles and how we can um increase diversity or even the opposite how we can increase personalization with regards to certain
Andreea Iana : Aspects or attributes of the news So we we looked at the very standard categories sentiments and so on and so forth But what we try there is to create something modular where we can say OK we will always care about the relevance part So whatever is recommended to me has to be relevant up to a certain degree because otherwise I will just not care about that
Andreea Iana : But then for example if a user wants more diversity with regards to a category uh but another one wants personalization with regards to viewpoints um how can how can we make that work um sort of on the fly uh and I think here the idea would be to have sort of a modular system where we can turn these knobs um on the fly so that we can really customize this to to different users and to different preferences
Andreea Iana : And this brings me back to the idea of before how can we make sure that someone doesn't read so that we don't overrepresent the conspiracy theories and so on and so forth Uh this would be the idea a system that we could use on a website or on a platform where we have some building controls that come from the editors or um
Andreea Iana : Yeah from other stakeholders and then the user can within certain limits turn this knobs So for example we also know that if you have too many options users get confused So let's say you have a default value for everything and then you have elections and you care a lot about politics then you want a lot more personalization So a lot to see the same sort of news within the time span
Andreea Iana : Um that are related to politics but in another time period of your life you would like for example a lot more categorical diversity to read about um
Andreea Iana : Yeah traveling music sports and so on and so forth
Andreea Iana : Um
Andreea Iana : And I think long term this would also mean that we should create something like that that brings together different aspects that we care about so not only the accuracy but also the diversity the
Andreea Iana : The language and so on and so forth so I don't think we can squish everything in in one training gloss or something like
Andreea Iana : this
Kyle Polich : We touched on news rec lib it's an open source thing we can link to your GitHub page in the show notes Uh what's your vision for the future of the library Who might consume it and where do you hope it'll go in the future
Andreea Iana : Yeah so actually a big update to the library is uh quite overdue so I'm um hoping to to focus a bit on this uh in the near future so
Andreea Iana : We already have a couple of data sets in there but there's um a couple more that are not already in the library that um we should be adding them as models So for example uh at the time that the library was released we didn't have all these um LLM based recommendation so that would be one side Another very interesting thing would be to also include more yeah approaches for diversification or um
Andreea Iana : Other methodologies and models that are not yet there so far I think it's mostly used in academia not that I know of industry using it but generally I think it would still be a very good place in the future for people to easily plug in their own model or take existing models and see how they compare against each other So it already had quite a bit of adoption for
Andreea Iana : I would say quite uh a niche domain but in the future it would be nice to to see it grow more and uh of course have also a lot more input from from other users and contributors
Kyle Polich : Well I'm curious if you have any recommendations for people interested in news recommendation who are earlier on in their career than you they like the field they wanna get into it it sounds like just becoming a heavy duty machine learning engineering person doesn't mean you can't contribute but that's probably not the optimal path What are the most promising areas for people to study
Andreea Iana : Yeah I mean having some idea of machine learning it's a very useful thing I think not only looking at it from the machine learning perspective but I think what helps a lot is um yeah coming at at this problem from different angles So um I was lucky enough to not be only in a recommender system group so I had a lot of exposure to natural language processing uh which helped a lot on coming up with ideas on uh actually how to simplify news recommenders and how to better represent news
Andreea Iana : Also of course the very related information retrieval field but going beyond that I think a lot of interesting perspectives especially if we talk about the responsible AI is also people with backgrounds in uh social sciences and here um I think yeah media and communication uh even psychology law is very interesting So how can we make all the systems uh comply with um all these regulations nowadays How do we satisfy all these different stakeholders
Andreea Iana : Yeah generally I think a lot of involvement also from the social sciences and humanities So while knowing machine learning helps build the system it's very interesting how how we make the systems work not just from the algorithmic point of
Andreea Iana : view
Kyle Polich : And what's next for you
Andreea Iana : So um as I said in the beginning I'm very interested in this uh responsible and inclusive AI for information access and the news recommendation was the beginning for me in my PhD but uh I'm generally more interested in also other
Andreea Iana : Other areas So for example now I'm working on retrieval augmented generation where we still have the classic retrieval side and the generation side And here I also want to see OK what happens when we try to add diversity in the mix So for example we ask a system a question and what happens if it retrieves more diverse sources Does it give us more diverse answer or does it represent these sources equally
Andreea Iana : So I want to see how methods that have already been developed for information retrieval by recommender systems can be reused or modified for these newer generative uh systems So this would be one thing in this work and another thing uh sticking to the multilinguality part as I've mentioned before we have a lot of languages that are not uh standardized um and don't have a written form
Andreea Iana : And maybe it's hard to work with some of these since they are not so easily reachable but a very interesting case here in um especially in Europe is that of linguistic variation in terms of dialects or different
Andreea Iana : Uh small regional changes in the language and these are exactly mirroring these spoken languages because these forms of the language are mostly spoken and they don't have a standard variant So for example if you ask a person from Bavaria to write down something from the dialect uh him and his neighbor probably will um will have two differently written versions And I think that looking at these sort of linguistic variations on a more fine-grained smaller
Andreea Iana : Scale would help us develop technologies that would also be applicable to all these languages that are out there and that are don't have a written form and this would um then hopefully in the long term uh bring us some technology that could help us also read or hear about events in uh from the native speakers uh where those events happen without using better machine translation or AI that helps us understand those languages
Kyle Polich : Well where can listeners follow you online
Andreea Iana : Yeah so I have my own web page just Andrea Aya GitHub.io and I generally post a couple of yeah news about talks papers um that I've published projects and so on For people interested in publications um there's always Google Scholar just looking for my name uh and I'm also sometimes a bit active on LinkedIn X and Blue Sky That's mostly it
Kyle Polich : We'll have links in the show notes for listeners to follow up
Kyle Polich : Andrea thank you so much for taking the time to come on and share your work
Andreea Iana : Thank you very much for having me and for the interesting discussion Definitely