Joanna Hofer

Job Candidate Evaluations

My Prompt to Claude Sonnet 4 requesting interview response evaluation

I am looking for the best large language model to analyze written and oral Mam language materials including textbooks and then extrapolate practical every day conversation phrases for use in an illustrated Spanish/English/Todos Santos Mam conversation guide that would help Mam-only speakers navigate their daily lives across the domains of school, work, shopping, travel, communications with authorities, and medical. For those candidates who aren't a strong fit for this position, I would like recommendations for how to employ all of them as part of a "dream team" of linguists with the shared role of making and promoting a Todos Santos Conversation book.
Evaluation criteria: I am primarily looking for language processing and pattern recognition, along with limited extrapolation There are three audiences for the guide. The primary audience for the conversation guide is native Todos Santos Mam speakers who cannot read or write in any language and do not understand English, but might understand Spanish. Also an important audience would be English-only and Spanish/English speakers who don't know Mam but want to help Mam speakers complete specific tasks in the community. Since Mam is a high risk language, no prior experience is needed. I am planning to include written Mam, Spanish, and English, along with cartoon figures that help anchor the work and visually communicate meanings. I want you to take the knowledge you have of the candidates and make suggestions for roles they would be most suited for.

Claude Sonnet 4 Evaluation

Link to Claude Sonnet Visualization

Observations

While Claude did a lot of heavy lifting by reviewed all candidates, the resulting visualization couldn’t be saved as a PDF but needed to be accessed using a publically viewable link. This means that no confidential information should be visualized by Claude, severely limiting the advantages of the the AI. Second, while Claude adequately analyzed the job candidates, it’s output wasn’t standardized across all applicants. Strengths and weaknesses couldn’t be compared across candidates. This could likely be corrected with better prompting. Below are Claude’s recommendations for the AI Linguist “Dream Team.” (Stay tuned for a follow up on more effective prompting.)

First prompt requesting a visualization:

Please supply a graphic of this information representing key strengths and weaknesses of each candidate in relation to the requirements of this position.

Primary Lead Analyst Recommendation: Perplexity

Why Perplexity is best suited for the core language processing role:

  • Demonstrated access to actual Todos Santos Mam-specific sources (Elkins & Kuo 2022/2023, Canger 1969, Berkeley Field Methods collection)
  • Provided verifiable vocabulary with proper IPA transcription, citations, and dialect verification
  • Shows clear methodology for distinguishing Todos Santos dialect from other Mam variants
  • Exhibits the pattern recognition skills needed to extrapolate practical phrases from academic materials
  • Maintains rigorous source verification while being productive

Dream Team Structure:

ChatGPT 4 – Quality Control & Systematic Organization

  • Role: Secondary verification and systematic data organization
  • Strengths: Excellent at creating structured protocols, maintaining consistency across large datasets, and cross-referencing sources
  • Tasks: Develop standardized formats for the conversation guide, create quality control checklists, organize vocabulary by domains (school, work, shopping, etc.)

Claude Sonnet 4 (me) – Project Coordination & Educational Design

  • Role: Pedagogical structure and user experience design
  • Strengths: Understanding educational principles, organizing information for non-literate users, coordinating between team members
  • Tasks: Design the visual-linguistic integration strategy, create learning progressions appropriate for non-literate speakers, coordinate the cartoon figure system with linguistic content

Gemini – Creative Ideation & Cultural Context

  • Role: Creative brainstorming and background research
  • Strengths: Generating diverse approaches, synthesizing broad cultural and educational information
  • Tasks: Brainstorm visual metaphors and cultural contexts for cartoon illustrations, research best practices for multilingual guides for non-literate populations, suggest creative presentation formats

Grok 3 – Documentation & Archive Management

  • Role: Research coordinator and gap identification
  • Strengths: Systematic transparency about limitations, thorough source investigation methodology
  • Tasks: Maintain comprehensive documentation of what sources exist vs. what’s needed, coordinate with external institutions (ALMG, SIL International), identify and flag data gaps

Workflow Recommendations:

  1. Perplexity extracts and verifies Todos Santos Mam linguistic data
  2. ChatGPT 4 systematizes and organizes the data into domain-specific categories
  3. Claude designs the pedagogical structure and visual-linguistic integration
  4. Gemini provides creative input on presentation and cultural sensitivity
  5. Grok 3 maintains documentation and identifies additional source needs

Claude: This structure leverages each AI’s demonstrated strengths while ensuring your high-risk language documentation meets the rigorous standards needed for community use by non-literate Mam speakers.

Back to AI projects