Souls in our systems ....

Moss: Hi, welcome back to Going Human. My name is Moss, and here we go, episode two, Souls in Our Systems. I'm just gonna set a stage for you first today, and I'm gonna put you in a room. You're in the emergency room of a hospital. An ambulance just wheeled an elderly woman into the triage room. She's in critical condition, unconscious, and her blood pressure's crashing. The triage nurse is hooking her up to things, and as soon as she gets her hooked up, the screen lights up and says: sepsis, sepsis, sepsis. And the charge nurse reacts exactly as she's supposed to, on script. She says, get her in a bed now, fluids, stat, all those hospital things people say when they're in hospitals. But then the bedside nurse, the one who was there when the patient was first rolled in and who's been sitting by the bedside the whole time, says: no, no, no, look, there's a dialysis catheter. This is a person who's got kidney failure. So I don't know what you know about kidneys, but kidneys, allegedly, are the organs in our bodies that filter all of our fluids of their toxins. And when you have kidney failure, those fluids don't go through the kidney. They just back up. So the bedside nurse knows that when your fluids can't go anywhere, they drown you. He said, we're not going to put this person on fluids right now, because she'll drown. And the floor nurse who said get her fluids said, no, you don't understand, we have to do it, we're saving this person's life. And the bedside nurse said, no, we're not. There was a doctor nearby who heard the argument, and the doctor said, whoa, what's going on, people? Turns out that bedside nurse was right. The doctor ordered some other medications to help with the blood pressure, and the patient was saved, not killed. So I just want you to sit with that for a little bit, because that is the premise of today's show. The machine was certain, the nurse was right, and the patient's alive because of a human. One human in that room refused to do what the machine was so certain they should do. So back to the scale. I know this is very analog, but in the old days we had scales, not even my old days, my grandfather's old days, but you get what I'm saying. The heavier thing weighted to the bottom, and that was the thing of most value. I don't measure AI on what AI can do, or what it will do, or what it can hope for. I really measure it by how it's affecting humanity. What is it doing to us? Toward each other or away from each other? Are we looking in each other's eyes, or are we looking in our own glass? So that's what today's show is about. It's about what's in a system that a soul can make sure is working the way it's supposed to. And that brings us to our Stone. Thank you so much for weighing in last week on our show. I've got some news for you at the end of this, but we're gonna jump right in with our Stone this week, which is the sepsis model. I'd like to start with beat one, just to share something personal. I was born into a medical family myself. I'm very passionate about medical systems and medical care, because my father was probably among the more famous OBGYNs in the country in his younger years. He was a diligent fighter for women's health care and a proponent for reproductive care, including birth control management and abortion rights for women, at a time when women were dying by the thousands in the streets trying to figure out their own care. He was a brilliant, kind man who sat with his patients for forty-five minutes no matter what, and who remembered everything about each one of them every time they came in. My mom, when I was eleven years old, went back to school and earned her master's degree, became an RN, and then she was my dad's OBGYN nurse practitioner in his office for over thirty-five years before they retired. Mind you, my mom did this because she was a charge nurse herself as a young woman when she met my father. And she also, with my father, had six kids. The youngest of six meant I was eleven, then I had a sister twelve, another sister thirteen, a brother fourteen, a brother fifteen, and then another older brother who was twenty-two. My mom had her hands full. I tell you all that because I want to illustrate a point a few different ways through this podcast. My mom sat me down once, after I had gone to the emergency room myself. I had some intestinal problems as a child, and I said, Mom, why is the doctor not in the room with us? Why is it always just me, you, and nurses, and then way at the end the doctor comes in? She looked at me, and I'm gonna paraphrase a little, but she said: you know what, honey, if you want a fringe diagnosis, an academic approach to the rubric of figuring out what's wrong, ask the doctor to come in the room. But if you want care, lean on the nurse. And I never forgot that, because she's right. It's the witness, it turns out, that's the power in predicting outcomes in care, and not a machine, and not even an educated third party who's getting sampled information like a doctor does in that situation. Nurses, they never look away. They can see every contraction in the delivery room, they can see blood pressure, they can see sweat on the brow, they can see panic in the eye. They see it all. And they bring all of that context in, and when the doctor comes in, they're talking in shorthand because the nurse knows what's going on. So I just want to make sure I put that down in beat one, because it's really important. The nurse stays until the head crowns. The machine just takes a sample. So whatever ruled here, the Epic sepsis model, sort of rides in the chart. Michigan checked it. I've got some sources on this. It missed two out of three in judgment and cried wolf on one in five. This is a 2021 study, still running a hundred-plus organizations. The fix was slipped in quietly at the time, but in a lot of ways it's still really a black box. We still really hope the sepsis model is helping more than it's hurting. At the time, 33 percent was the model's success rate. So that's not a flawed tool. That's a coin flip in a lab coat. It's not really getting us to a closer standard of care. And that's obviously what a hospital is there for. But before I continue, I just want to point out the obvious, which is that standard of care in hospitals is more and more about efficiency and less and less about touch. And in order to do that, you have to have systems, functional logical systems that can help relieve some of the burden. As we talk today, and as we bring on our guest, I want to be very explicit in saying: the people who own, manage, run, develop, and deliver the systems we're going to be talking about are not the responsible people for the outcomes of what the systems are delivering. They are the carpenters. In many ways they act as architects, but especially in healthcare, the interest groups that drive these systems are variant and multiple. So oftentimes you have people arguing about what's really going to make a difference, and some are pushing things that are sacred cows for them but really aren't defensible in some instances. There's a lot of dissonance in the ways these systems get defined and designed. And then it's the carpenters who have to nail them all together and make them work and bolt them on and get them to optimize. So I just want to make sure I say that as we talk to our expert today. We're going to be hearing all of the shenanigans that go on when these systems are being built, and those are the faithful soldiers who are really keeping the disasters at bay. So now I'm gonna jump into a different personal angle, which is my operator angle, because I've been in the room when these automate-or-not decisions have been made. I haven't been in the room when we were saving babies, to use my dad's phrase, but I've been in the room when we were doing things like speccing out miles of conveyor belt in a very large warehouse for, say, a company whose name rhymes with Isney. When you're speccing out that conveyor belt, you're relying on this 80/20 rule. So 80 percent of the time everything's gonna go great, and up to 20 percent of the time it can go bad, but we can live with it, because we have remediation lanes and all that other stuff that makes the things that can't get put in the right place have a little depot to go to, and then we can pay one person to triage them. In the meantime, 80 percent of our packages have gone through with no problems. So we make those decisions in automation all the time, but we do it through the lens of widgets. If a widget falls on the floor, we sweep it up. If something we're automating severs an arm, that's not an 80/20 decision. Eighty percent of the time no severed arm, twenty percent of the time severed arm, probably not gonna work. So I like to say: you automate the stray widget, you don't automate the severed arm. I know that sounds funny, but it's kind of funny, right? Don't automate the severed arm. That's your lesson for today. I see this sepsis model as kind of a severed-arm application. It's shipped like a widget. People are told to follow it. They're literally listening to the voice of a hospital tell them what to do in somebody's care, based on an algorithm that's sitting behind a bunch of data. One more thing I'd like to point out: the sepsis model is not pure AI through the lens of Claude and ChatGPT and the things we've popularized as our avenues toward understanding AI. It's essentially a predictive analytics tool, and it was built into the Epic EHR. The model itself is called the Sepsis Prediction and Optimization of Therapy, or SPOT, model. Epic also called it the Epic Sepsis Model interchangeably. Its core mechanic is that it's continuously running in the background, scanning data from the EHR, the electronic health record system, every fifteen to sixty minutes. Intervals vary by hospital and how it's configured. It uses vital signs, lab values, nursing flow-chart data, medication orders, demographics, comorbidities. See, I'm human, so I can't really pronounce things very well. The score is generated using a logistic regression model. It's not deep learning in the sense of machine learning systems, but it is trained on Epic's large multi-institutional data set, which weights the inputs and produces a probability score. Once that probability score hits a certain threshold, that's what tells the floor nurse who's running around to sixteen different beds, people in different states of criticality. She sees sepsis and she makes her order. Sepsis makes the order. She doesn't have time to sit there and go, okay, somebody told me what to do, and I've got fourteen other people here who are redlining, but this person needs something, and the screen's telling me what to do, so I'm gonna do what it told me to do. The thing is, while this may not be AI, it is the bedrock of some of the AI technology and what led to it. It stands to reason that if we're in a world where AI is increasingly rewriting papers for us and helping us with our thinking, the next step for the sepsis model could very well be a rational layer that sits on top of this Epic model and says, hmm, okay, we're one step ahead, we can do something a little bit better with this data than the original model did. So this is a discussion we're here to have. Key criticism, and I realize this is some old information, but like I said, after the old information came in, some adjustments were made that improved the model moderately, but the underlying scaffolding remained, and there's still a bit of a black box. In 2021, a study published in JAMA Internal Medicine, the peer-reviewed journal, found this model had poor sensitivity and specificity in real-world use, that it was missing a significant portion of sepsis cases while also generating high false-positive alerts. And in a clinical setting, when you have a high volume of false-positive alerts, you're also generating higher false-positive alert fatigue. After a while, people stop listening to what you're telling them to do anyway, and they just insert their human touch. So you're creating an environment of irregularity by trying to mechanize and automate something that you're trying to make consistent. False positives, not good. False-positive fatigue, even worse, because it literally does the exact opposite of what you're trying to create. I also want to make sure to say the model doesn't diagnose sepsis in a clinical or legal sense. It just flags the elevated risk, and then the clinician has to make a diagnosis. But remember, these are human beings on these floors. They went to school, some of them for decades, to get the credentials that allowed them to do the care they're passionate about and deliver to people in real time every day. When you tell somebody, we don't always trust you, and we're gonna get to the answer faster without you, just do what the screen says, you're starting with a rolling eye. That person at the other end of that directive is saying, really? You've got to be kidding me. And then when your model says sepsis forty times but only about thirty-one of those times was it accurate, that person's eye roll turns into, I'm not gonna trust this at all. So that's what happened in that hospital that day. We had one nurse intent on following the algorithm, and another who said, uh-uh, I have other information that the algorithm isn't seeing, with their own eyes. So think about it. Missing the sick, it'll invent the well. And both times, the only thing between you and that mistake is a human who might know you. This beat is more about some of the other things AI is directly doing. Many of you have gone to the doctor lately, and a nurse has come in before the doctor and said, I'm gonna have my records management system record this and we'll write up the care notes we talk about. Is that okay with you? So you consent and you continue with your exam. The doctor does the same thing. It's actually a bit of a miracle in a lot of ways, because it means the doctor isn't typing and menu-hunting and doing all the things they were doing on their very elaborate systems while they're trying to hear your story. But it's also taking transcription. That transcription has to transcribe voice into text, and then the AI model takes that voice-to-text transcription and turns it into meaning, helping it understand what's happening. And it is not uncommon, and has not been uncommon, for these systems that are doing the scribing to mischaracterize things that are said, because it misses body language and many other conversational cues. In some extreme situations, parts of conversations that never even happened got snuck in, because of the lack of guardrails in some of the AI systems, especially as they were first being implemented. I illustrate that because it's an example of something peeking into healthcare management and administration that stands to really help remove the friction between care and documentation. So here we are, back at the scale. Everything aimed right, well rationed, well reasoned, well guardrailed, it'll save lives, and that's real. If it's aimed wrong, or the authority didn't get earned, or there's no witness in the room to say no, it's not gonna happen. So there's a little bit of hope here. There's a federal commission, MACPAC. I'm sure I could find what that all means in a note somewhere, but I won't bore you with it. This is a ruling, it's not a law yet, but it's a recommendation that will get introduced, that basically voted unanimously to keep the human directly tied to the no. And in this case, the no was Medicaid and federal access to medical care for poor people. People were getting denied by an AI decision rubric and just told go away. MACPAC found that the error rate on these no's was catastrophic, and that the incidence of an accurate no was far below its inaccurate sense. It became very clear that the yeses were strongly filtered and well qualified, but the no's were very badly judged. So again, we've got a situation where our systems are telling smart people what they should be thinking, and they could be right, but they're often not perfect. Are we a widget, or are we a severed arm? And where is that person in the room to keep us safe and sound? The fix isn't smarter machines. It's keeping a soul in the system, allowed to say no and trusted when they do. This is why I needed to talk to Jen. Jen Clifford is a fantastic soldier. I'm a new fan of Jen's. She and I met through a common contact, and I've had the great joy of being able to spend some time with Jen alone, but also in a social setting with many of her colleagues. Jen has spent twenty-three years in healthcare IT. She's spent eleven of those running teams that directly do product management and deployment for clinical content and workflow inside of Epic's umbrella systems. For me to say she is a soul inside of this machine is a true understatement. And I want to say, she is not the medical provider in the room. She's not that floor nurse who said no. But she is the person who interacts with the architects, the business requirements, and the output heuristics of what she delivers. She's a common voice. The people I met at her event are the common voice, they are the strength behind her institution's well-hewn guardrails, and it's just a delight to have you here, Jen. Thank you so much for coming. You know where the judgment's hidden in this software, and you also know what it can cost when somebody automates it away and it doesn't work. So thank you very much for joining us. I'd love to ask you: as you heard my introduction about the Stone and some of the information I shared, if you put yourself in that room as a human soul, and that was your mother, and you were just a fly in the room but couldn't talk, and you saw your mother wheeled into that hospital, and you know she had that kidney dialysis catheter in her collarbone because you're somebody who probably has to flush it three times a week to make sure it works. When you saw her wheeled into that room and you saw the interaction, and you were floating in the air, what were you thinking? Jen: I think what I was thinking, at least when you're sharing the story, that is true. That's reality. Whether it's information coming from the machine where you make the decision, or it's just the information that I know as an expert, there's still a judgment call. And you have to do what makes sense. It can't just all be what the paper says, if that makes sense. It's really different when you're not in that IT space and you're actually in that room with your loved one, because I've been there, both with my mom and my dad. It's very different. And when I look at it, because I work in IT, I see that computer there. I want the healthcare provider interacting with my loved one and taking care of them. That machine is secondary. And you talked about that ambient listening concept, where you go in and meet with your provider and it documents the encounter for you. We've had electronic health records for many years, and it's all been about that IT. In my mind, just watching it unfold over the years, it has created a distance between the provider and their patient. What I'm seeing with this artificial intelligence layer is it's giving us an opportunity to bring that relationship back. My provider can focus on me. They don't have to worry about the ten million things they are required to document. Because you have to document it in a very specific way so that predictive models actually work and pull the right information. So it's about how the information goes in. Moss: Right. Hold on. My cat's automatic feeder just went off, and my cats are gonna jump through the screen. Okay, I'm sorry to interrupt. We have to feed our cats. It's gonna make this sound for the next five seconds. Hold on. Jen: Well, they should jump through the screen. I'm surprised I don't have any here. Aw, look at that. Yeah. Moss: Can you see him back there? Faithfully waiting by their automatic cat feeder. Okay, one more time, here it comes. Last one. Wait for it. Did you hear that? Okay, I think that's it. Jen: For what it's worth, I don't really hear it. Moss: They're so cute. I love my cats so much. Okay, wow, that was a bit of a departure. So, Jen, what were you saying? You're heralding the advent of presence in a room by a caregiver, in the form of a professional who you have to trust, who you get to have, but whose attention is divided in those moments. This scribe-level application of AI can be a really brilliant way, as you say, to bring that connection back together and to have the provider stay in the room. Jen: Yep. Yep. Moss: Tell us a little bit more about your journey in EHR, and with the system specifically, and what are some of the things that have come across your desk over the years? I should point out that Jen is now at a place and a level in her career where she's earned the badge of team ownership. She's the person who grabs talent from all over her network, and every other person's network that she trusts, and she qualifies them and brings them in, not only to her own organization but to other organizations that are satellites to her. So her interaction model day to day is more about making sure that the people who are the souls in these machines are thinking about it with a passionate lens, and making sure their best foot goes forward in the delivered results. So just want to make sure I said that. But Jen, help us out with some of those things that have come across your desk in the many years you've been dealing with this. Jen: Well, this is actually more of a personal story, but it kind of set me on the road of trying to think about how do we make our electronic health system add more value to what we do. This is a story a friend told me years before I met her. She ended up being diagnosed with pancreatic cancer, but she got ahead of it and survived. It was caught early. If you catch pancreatic cancer early, you can treat it. But most of the time it's progressed so far that there's nothing you can do. She had noticed some symptoms, and her partner got her a Groupon. Remember Groupons back in the day? That's how long ago this was. Moss: I love where this is going. Did it involve a mani-pedi? Jen: A naturopath. So she went and did this visit, and the naturopath got really curious and said, give me your health records, everything you've been going through, I want to look through all of that. So she's got this big stack of paper. This provider took the time to go through everything, and then called my friend back and said, hey, I believe you are showing symptoms of pancreatic cancer, and you need to deal with this immediately. And so she did. And my thought was, how many providers out there have the time, even have the time that they can take, to go through all of that? That's what we want. When we pay for a provider and we pay for healthcare, that's the level of detail and expertise you want. But the reality is we have to see many patients, you have a schedule, you have to end your appointment so the next person can come in. So it starts getting really prescriptive. In the system I work in, that is very much a model. You have certain lengths of appointments and you have to meet that. And you're measured on that. Metrics drive behavior, so you're measured by that. So how close of a conversation do you actually get with your provider? I am also in IT, and I know how incredibly difficult it is to build tools that can actually mine that data between disparate health systems who may capture it in a different field name, in a different structure. So how do you then marry that data together to make any kind of assessment? That's where I see AI coming in. It can actually do that, where you can take unstructured data and make sense out of it. But to your point, how accurate is it? And then when you start having your AI looking at that data, when it starts to degrade its model and start looking back at what it told you, how do you as that human know what to look out for to make sure it's accurate or not? So I've kind of gone a little circular there, but it's important for us to figure out and make sure it's not giving us the wrong thing. Most of my IT work is not actually in the building of the systems. Like you said, I'm leading and managing the people, and I'm trying to understand what those problems are. I have to then sell my ideas to leadership at times. That's an area that has always been a struggle for me. You've got the format, you say X, Y, Z, or an SBAR for example. So there's a certain rubric. But you still have to put that down on paper, and when you're in a leadership role, you usually have fifty million other things hitting you. So it's hard to concentrate and think. So I have been absolutely abusing the hell out of Copilot, and I have it write those documents for me. It can do it in such a clear, crisp way, it's accelerating my ability to have decisions made. And I think that's gonna be one of our biggest barriers. We've got so much information coming at us, how can we as humans not be the bottleneck, but actually ingest it, use it, and move forward? Moss: Yeah. And I really thank you for bringing that last bit, your Copilot experience, and the benefits it's bringing, because there is this, I'm sorry, there's a cat about to jump. I need a recording studio, that's what I really need. Jen: No, the cat's perfect. It adds that perfect touch. It makes you human. Moss: Our ability to deliver decision support in our own minds, just to organize thoughts when we've got so much going on, becomes really hampered. And I'm the same way. It's not like I have AI writing my emails for me and I just don't look, but I spend a lot of time documenting my own software systems myself, asking my AI, Claude, to go through code and document the code that's running my production system. Now, Claude is generating almost 100 percent of the code that it's documenting, but Claude is documenting it accurately because it's from code. So in a lot of ways, while I'm a non-coder relying on Claude to help me write the code, I'm relying on Claude's summary of what the code is doing, which is mechanical, to help me ensure that what I've built is what I've architected. So I just wanted to share. That's my dog, that is now getting antsy. I have a farm animal problem in my home. So anyway, I really loved that you brought that in. Do you have any, since we haven't said anything about where you work, do you have any gory stories you want to share? Even if it was from before your Epic days? Jen: You know, I think it's interesting that you're talking about the sepsis model. A few years ago I managed a team that was responsible for rolling that out in the EDs. And I really should be careful with how I say this, but we are an Epic shop, and we actually chose to roll out an internally developed version of it, because our providers did not trust the outcomes of that Epic model. It was too new. Today I think it's much better, probably a lot more accurate, but at the time it was not giving us what we needed. And it gets to that issue of all the different systems capturing the information a little bit differently. So how do you pull that all into a predictive model to do that analytics? Moss: Yeah. And I love what you said about the tools that are gathering the information, and this idea of slugs that help you point to a specific table, column, and row in a database that is the harvestable data these models use, that will get us from, I don't know what the Sam Hill to do, to here's your decision. Getting that slug mapping to happen not only intelligently but with 100 percent accuracy is the underpinning of all of this. So it's really interesting. Hold on one second, I'm gonna see if I can do something about my hellion. Hold on. Jen: I hear the kitties. Oh my god. Moss: I've got a little pacifier thing that wedges in his crate, specifically in his exact manufacturer crate, and it makes it so he has something to keep him entertained. We're gonna see if that works. Okay. Shut up, dog. Okay. That wasn't dog abuse. It was dog management. Just so we're clear. So I'm gonna back up. I'm sorry I interrupted you, because you were on to something there. Okay, Jen, bring it back. Tell me again what you were just saying about Epic, and the fact that it's a system you're working on and where your focus is on it. Jen: So the predictive model. The cool thing about Epic is they're constantly developing and taking feedback and reiterating. But oftentimes the first version of what we'll get from them, we've got to work through bugs. So probably around the same time as the study you quoted, I would say we were looking at implementing that model into some hospitals that I'm responsible for. And our providers were not really bought into that model because of the accuracy. We have our own internal, we're a very large organization, so we've got a data and analytics team. And we actually did our own homegrown sepsis predictive modeling in one of our markets. The market I was responsible for, we opted for ours, we trusted it more. That is changing. Epic is getting much better, and we have different technology available. So a lot of that discrete data mapping is still important today, but we're developing more and more tools that can look at unstructured data, connect it together, analyze it, and make decisions. Moss: Yeah. Great. The point of the cast, and the glory of your attendance on it today, is really the discussion, and it's just warming to hear a logical brain like yours articulate the value, and acknowledge some of the cracks in the pavement that are a part of IT evolution. Things don't get better because we're afraid of them. We have to meet their challenges head-on. In a world where we want humans to matter less and less, we're not paying people what they deserve or need to live, and we make money and advance our own financial aspirations in ways that are often very singularly focused and have nothing to do with the people who are left behind when these things are delivered. So culturally, we in IT are often faced with taking problem spaces and finding solutions for them. That's a cattail, by the way, if you're watching this on YouTube. So the distinction is an important one. The builders, us carpenters out there trying to take your business requirements, even us architects who are understanding problem spaces and building solutions around them, we want nothing more than to do something that is wildly brilliant and perfectly optimal. And just like the person on that retail distribution center floor that has miles of conveyor belt and is dealing with that widget that fell off the belt, in IT we're faced with our own resource limitations, where we've got multiple stakeholders often arguing about what it is we're to be delivering and how it's going to come out. So there's a lot of juggling that happens. And it's just really great to have your perspective on the show, Jen. Really love it. Jen: Thank you. Moss: Here's how I would land it, and then I'm gonna ask you to tell me what you think. The machine should get the busy work. Jen: Agree. Moss: The work that is not adding any human value. A doctor who decides where to put five words in a field on a screen so that he's compliant for all the regulatory needs of every single chart that goes through every single system, that is busy work. Now, does that guy get to think about what's happening with somebody's chest when he's listening to it on a stethoscope? Yeah, he does. But if 60 percent of his attention is worrying about slugs and what data goes into which field, we're no longer talking to the provider at the same level we were when we didn't have the system. So yeah, the machine gets the busy work, but never the authority to overrule the one watching the patient. How's that land with you? Jen: It lands really well. I one hundred percent believe that. I want the provider to validate whatever the machine heard and put in there. I want them to say that it's correct before it gets logged in perpetuity in my electronic health record, in my personal record. And I think every human should feel strongly about that. It is challenging to get the wrong diagnosis removed from your record. There's just a lot of work you have to do, working with different departments, et cetera. So you want it right. Moss: Well yeah, challenging over time. But what about the wrong diagnosis when there's no time? It'd be great, you have a cold, no, it's a flu, okay, I'll take six months to remove the flu diagnosis. But it's like, you have pancreatic cancer, oops. So while it's comforting to know that even with a lot of effort you can change a diagnosis that's wrong, you really want the diagnosis to be right, and you want the speed to diagnosis to be lubricated. That's what we're hoping. The speed to an accurate diagnosis is going to require a really careful dance between technology and human in order for us to optimize healthcare. Okay. Well, an AI that hands you an actual human instead of a glowing rectangle. This is the first in a long series of Fuckery that we're gonna go through tonight. As my audience knows, Jen, we have what I call the Maelstrom. This is the crazy part of AI in the last week, where I pull some stories and I expose you to an opinion about them and ask you for your opinion. Basically it goes like this: fuck yes, fuck no, and fuck maybe. The fuck yes and the fuck no, that's all just you and me riffing. But on the fuck maybe, we're gonna riff, and at the end I'll point everybody to my website. The website has the ability to take a vote, your thumbs up or thumbs down, fuck yes or fuck no, and add comments. From time to time I'll read the comments, but every week I'll give everybody an update of where we landed on the fuck maybes. So the fuck yes is a beautiful one, and there's gonna be more and more of these coming out, the ones that either warm your heart or make you think we're really in a place where things are gonna change positively. So, Eldera. This is a system that matches people over sixty with kids who are somewhere between fourteen and eighteen, I can't remember exactly, to just have conversations. This is AI that finds ways to match people with specific interests into conversations that create a connection and an interaction model based only on humanity. It's just about the facilitation part. They do these calls weekly, and ninety-four percent of these matches continue talking well over a year. So this is AI that's been specifically targeted toward the loneliness epidemic. It's taken those who are probably the most vulnerable in loneliness and matched them with people who have the most energy, who have enough going on so the conversation can keep flowing and new things can come to people. If I'm sixty-eight years old, don't ask me how close I am to that age already, and I talk to somebody like you, we just geek out about what we geek out about, and our friends sit around us and go, God, won't they ever shut up. Well, Eldera does this in a way that makes it so these older people get to ask kids what they're doing these days. So that's my first fuck yes. I wanted to ask you what you thought about it. Jen: I like it. And I like it because it is bringing people together. So it's an actual relationship and connection. I resonate with that stuff. Moss: Yeah, me too. Okay, so now for the fuck no. We might even swear more during this section, Jen, I don't know. I've got a potty mouth. You're probably all self-conscious about that for professional reasons, but I really like this one, because I live, die, and breathe this all day, every day, and I work hard, ten, fifteen hour days most days of every week. And I work with Claude. Claude is a product that comes from a company named Anthropic. Claude has three anchor products. One is chat, that same interface you think of with ChatGPT, where you ask it to draw a picture of a dog farting in August and all of a sudden you get a weird picture. It has a thing called Claude Code, which is the equivalent of Copilot on the ChatGPT side. And it also has this thing called Cowork, which is a way to build workflow automation and scripting into repetitive tasks using a skills paradigm. So I live and breathe by this technology all the time. Well, last week, a week and a half ago, Anthropic released its most powerful model ever. And it was incredible. When I used it, I was like, my God, I could take over the world with this thing. It was only live for about a day and a half. I used it for that entire time, because I was using the product anyway. And then they shut it down. You went and looked at the little why-they-shut-it-down thing, and it was all, you know, the government said we're shutting it down, this is what we're using to defend ourselves, this is how we're talking about it as a company. But really what happened was a hacker was able to crack the thing, and they got the guts of this AI prompt downloaded and exposed it to the world. So a huge regressive leak of information that could be very volatile. In a world where we're not governing AI and its output, this got very scary. The reason I'm putting this in the fuck no category, Jen, is because Anthropic was the parent in the room. Jen: Mm-hmm. Moss: Anthropic is the company that came at AI with as much altruism as you could possibly hope for, in a realm where everybody's racing, hopefully not to the bottom, but to be the best. And oftentimes people end up racing to the bottom because they just want to be the first with the best and greatest. Anthropic was intoxicated by this. They went out a little too soon, in my opinion, and they didn't lock their stuff down, so a hacker was able to expose a vulnerability and take advantage of it. I'm glad the government pulled it, given what I know about the situation, but I wanted to ask you your thought. Is this a fuck no for you too, or where are you on it? Jen: Yeah, I'm with you. I think it's a fuck no. There has to be responsibility around this technology. These tools are so powerful. I'd say fuck no. Moss: Yeah, good. It's not a hacker story, really. I'm gonna have to let my dog out, hold on. It's not a hacker story, it's an Anthropic story. It's a startup story, really. And to your point, these systems are very powerful, and we can't keep our own hand on the levers of these systems if we don't have some guardrails on them. And I've always been a big fan of regulation, and of somebody above the people who are making money making the decisions, because putting the people with the pocketbooks in charge of their own safety is a dangerous proposition. So the responsible people in the room have got to stay responsible. Anthropic, this seems like a place where you might have dropped the ball, and an important one. I regret that it happened, and it would be really heartwarming to hear that somebody caught this and it got out at a time when you could contain it, and you can still release that model with the right guardrails on it, because it really is a strong delivery. It was really, really good. Okay, so here's our fuck maybe for the week. Are you ready? So you know that guy, Bernie Sanders? I love me some Bernie. I think he's a really good guy. He's one of the few on the Hill in our great nation who just tirelessly thinks about the impacts of the real humans powering America, and who's not bought and sold from a corporate perspective. Well, he came up with this idea that if over time America could own 50 percent of each of the big players in AI, we'd be able to create an accountability framework that would essentially make us shareholders. So as shareholders, we'd be able to say where AI is going. That kind of sounds good, right? It's a very populist message. I'm gonna ask you about that message. But first I'm gonna tell you that, like a week and a half later, one of the other politicians in our government today, whose name rhymes with Rump, came out with a proposition that it should be ten percent. And in his own words, it's a beautiful thing. But of course there was no detail, no rationale, he just likes to make sound bites. Bernie put a compelling proposition out there with some accompanying rationale. I'm wondering what you think about that one, Jen. This is our fuck maybe. Where do you think this goes? I've already sort of shared mine, but I could go either way. Jen: Honestly, I have never thought about this, so I've got to wrap my head around it. It's a different kind of concept. I do think that if we are invested in something, and, well, I'm not gonna take that back. It depends on the person. So for me, I like it. I want to be invested in my future. I would want to be able to influence responsible AI. And what I mean by that, it's not even just using the technology itself, it's how we build this technology. Because we've heard of all the things happening with data centers and water quality and pollution. How do we get ahead of that? So from an investment perspective, that's where I want us to consciously be thinking. How do we make it clean? How do we make it renewable? It's not going away. So you can say, well, I don't believe in what they're doing with data centers, so I'm just never going to use AI. You're probably using AI today and you don't even know it. It's not going away. It is not going back in the bag. It's out. So what can we do? It's true for just about any technology. Think about TVs, remember, you put them in the landfill and they're causing problems. So we always have to be conscious of how enabling our society can potentially damage it. So how do we avoid that? Moss: Right. I love that you brought up the TV thing, but I'd even go one step before that, to single-use plastic bottles. We haven't figured out anything. What we've figured out is cravings for more. We know how to get people to crave more than they have, and we found ways to inject into these cravings so people can feel satisfied. But the negative outcomes, the danger points inherent in anything that's novel, those guardrails have to be thought of and erected first. So from that perspective, I couldn't agree with you more. Now I'm gonna argue your point a little bit. Let's just say you and I own a fifty-fifty share of a widget company, and you're an environmentalist and I'm a capitalist, which in America, especially in their most extreme senses, are diametrically opposed to pretty much everything. And at the end of the day, we have a board of directors sitting behind us, and we're a public company. That board of directors has a fiduciary responsibility to only make money in a public company. This is not a B Corp situation. So we could be sitting there talking to them, and you could say, ooh, the point of environmentalism is the TVs, remember when we did that and then the fish started dying in the marshes around the dumps. Well, so there's that. And then I'll be like, well, yeah, but we can get fish from anywhere in the world, we gotta deliver this stuff because your stock price is gonna go up forty dollars per share overnight if we do. And when a board of directors hears that, and they know they're bound and sworn legally to a fiduciary responsibility that says they can only have one consideration, the bottom line, guess who wins? Jen: Right. Moss: So on an ownership level, I'm just gonna, and I know this is probably not professional, hold on, he's going to join us for the last portion of this show, I'm not going to be able to finish it. This is Kodak. Jen: This is the new professional. You have to have a pet in your video. Moss: You have to have a pet, it's a whole thing. That's a microphone, honey. So the point is ownership. My sense of this is that the point-counterpoint, the fiduciary responsibility thing, all of that stuff that's driving us, in my opinion, is improving stock prices and growing our GDP, or has up until very recently. All that stuff is happening, but it's not good for the planet and it's not good for human beings. So giving that power to the people as kind of a socialist idea, so they have their thumbs on the scale of what drives tax revenue, that is a romantic idea, and I love it. But I don't want the government to be making decisions based on the profitability of a product that is so pivotal. In my mind, this is where my brain goes: make it all owned by the government so that it's not a profit thing. The government can take an abstracted view of value in terms of how it delivers to the populace, and say, you know what, if we get sixty percent of the people who meet these conditions, who are getting public assistance, and we can get them into houses and jobs using this technology, we don't need a profit from it, because what we're doing is creating a reduction in our tax burden as a country. Jen: Not only that, then you're also building up that societal value. People are happier. They get to participate. They're not just trying to make sure they have food on the table. They can actually participate in society today. I feel that way about internet. Internet should be just available, like a public service. Moss: Yeah, I agree. So the ownership thing for me is regulation and mandates. I say we have to find ways to exact and exert mandate-based objectives on AI that solve real problems, and that have nothing to do with Elon Musk's ability to wave a wand and do crazy shit. I don't care how cool it is or how inventive it is. What I care about is, we've got problems. We still can't recycle a plastic Coke bottle. We can't do that today. We generate millions of them every single day and we don't know what to do with them. Is there a body of the government out there trying to solve the single-use plastic problem? Jen: So have you, gosh, where did, totally. Moss: I think they've been shut down. I think that was maybe something that rhymes with Oge. But go on. Jen: I totally don't remember the details, we'll have to look at it. But a group of kids won an award for developing something out of tamarind, I think, that actually eats plastic. It eats it. Moss: See, I believe that children are our future, Jen. Teach them well and let them, I'm sorry, am I quoting Whitney Houston now? Okay, I'm not gonna do that. This is a podcast. We're being very informative. Okay, well, that's the Fuckery for the week. And I'm looking forward to getting my scoring system up and running this week. We had some problems with it last week. I'm having to do some bug fixes on my system that I've written to do all that. So in the next day or two you'll be able to vote. The vote on last week's, which was the robot doing the massage, the point of it was a robot touching you with metal warmed-up arms, and the counterpoint was access. What if people have traumas, or just wouldn't otherwise get any of the therapeutic benefits of massage because of their own response to being with a stranger? The overwhelming response was on the counterpoint side. The winner there was that it's probably more valuable for people to have access to it. Those who do want the personal touch can still adopt it, but to be able to have something that could effectively do a therapeutic massage and eliminate some of the trauma was something people really resonated with. So that was the vote there. Okay, so we're tying things down now. The last couple of minutes here is just our Tsunami and our sign-off. This week our Tsunami is, I want everybody to think about doing one thing they take for granted is done automatically. Even if you have to duplicate it. So if you have an email come in with an appointment reminder that sets up a Zoom or something, and you've got it automatically set up to put it on your calendar and insert the link, recognize that this week. Go in that one time you see it come in and just make that digital record yourself, by hand. And just recognize that there's something behind your machine doing something you might take for granted, or that you might not think is a big deal. But every single one of those things happening behind the scenes is adding up for you, and it's helping. What we can't lose sight of, and what we can't blur, is the accumulation of all these things, and the ways they become de rigueur for us. We need to be making these decisions consciously. The mundane example I just made, about the appointment and the Zoom call coming in your email, seems really mundane, but you don't even have to be as old as me, Jen's quite a bit younger, to know that not so long ago we had to do it all on paper. Jen: There was no busy searching a calendar. You were making phone calls. Who's available? Moss: No, are you available? Okay, I'll call you back after the nine other people I have the same conversation with tell me eighteen different things, and then I'll tell you. So yeah, do something manual. Anything that's automatic, do it manually and just say, wow, okay. Things have come a long way. This is my humanity. I'm applying myself. It'll just give you a perspective, and I would love to hear your comments on my website about that. So to tie it all down, we have a nurse who beat a system that runs half the hospitals in the country, because he was present and trusted himself. And you have that same instrument. So think about it. One person trusting their own eyes does nothing. A million of us refusing to be overruled by a confident machine, that's our Tsunami. And I think that's how we have to think about the evolution of these important and groundbreaking systems as we move forward. Okay, this is the end. I'm gonna ask you to follow and subscribe. That means a lot to me as I'm growing my base. Go ahead and vote on the government-owns-AI premise from Bernie Sanders. And send this to somebody who's stuck in a system. If you know somebody who's stuck in a system or frustrated by a system, share this conversation. This cast is about the conversation. It's not about shutting technology down or lifting it up. It's about thinking about how we're intersecting with it. I'm Moss. This is Going Human, because robots can't. Back next week. Thank you for listening.