The rapid adoption of generative AI chatbots has intensified discussions around “trustworthy AI.” Understanding how lay users perceive and evaluate chatbot trustworthiness is essential to keeping these discussions inclusive, making formal assessments user-centered, and revealing instances of misplaced user trust. To this end, we conducted an interactive online study with 254 U.S.-based participants who were asked to investigate whether a generative AI chatbot produced problematic, questionable, or unfair responses. Using a researcher-supplied probing tool, participants could freely interact with the chatbot and flag any issues. Participants engaged in 551 open-ended conversations and primarily sought to probe for issues and topics related to their everyday use. However, participants frequently failed to uncover the issues they expected to find. Overall, trustworthiness perceptions increased after the probing intervention, regardless of whether issues were detected, and participants with higher initial trustworthiness were less likely to flag problems in general. Our findings suggest that lay users may have the right instincts about concerns with generative AI, but are not well equipped to surface these issues on their own. We also find that trustworthiness perceptions can increase rapidly, even among initially skeptical users, likely driven by users’ tendency to treat outputs as deliberate and reasoned rather than probabilistic, and by their reliance on surface cues—particularly performance and utility—when judging trustworthiness. Our work complements prior work by illuminating how everyday users assess trustworthiness in generative AI, broadening debates on trustworthy AI, and highlighting the need for stronger guidance to help lay users accurately assess and calibrate trustworthiness.