Using Generative AI to Analyze a Text for Language Learners
When we ask learners to read an authentic text, we want to make sure that they will be able to make sense of it with a reasonable amount of time and effort. Assuming that the text is generally familiar and appropriate for their level of proficiency, you can make the text much easier to read by identifying words and phrases that are clearly beyond the level of proficiency that you expect of them and providing additional support.
Among the many ways in which generative AI can assist you in your work as a language teacher, these tools can analyze texts and identify words and phrases that might be challenging for your learners. You can simply gather information about the text or you can ask your AI assistant to provide definitions or translations.
I have been experimenting with ClaudeAI for analyzing and translating texts, for reasons that I will explain elsewhere. You can interact with Claude using a very simple Chat interface, through a more complex and powerful Console interface, or by setting up a Google Sheet with the Claude extension. Although you could read the extensive User Guides and explore the whole Prompt Library immediately, I would recommend starting with trial and error. Simply start entering prompts, in everyday language, and see if you get a satisfactory response.
Texts can be several pages long, but I recommend staying under 1000 words. If you find that the responses are ending abruptly and incomplete, you can also go to Settings and increase the maximum tokens to sample at a time.
ClaudeAI by Anthropic
Anthropic > Chat with Claude for free: https://claude.ai/login
Console > Use the more advanced interface: https://console.anthropic.com/dashboard
User Guides > Learn more and start prompting: https://docs.anthropic.com/en/docs/intro-to-claude
Prompt Library > Explore examples of what Claude can do: https://docs.anthropic.com/en/prompt-library/library
The following prompts are designed to demonstrate one way in which generative AI could serve as an assistant in language teaching by identifying words and phrases that might be challenging for language learners. Look at these examples and adapt them in order to create monolingual or bilingual support. In this case, imagine that the learners speak Portuguese as their primary language (L1+) and are learning English as an additional language (L2+).
The following examples got good results from Claude, using the Workbench (https://console.anthropic.com/dashboard). As of spring 2024, several different models of Claude are available. Sonnet is the most advanced (the “smartest” if you will), followed by Opus and Haiku. I adjusted the settings to use the Claude 3 Haiku model (the least powerful and therefore least expensive of the paid models). “Expensive” is relative – in a few months of casual experimentation with fairly lengthy texts and a wide range of prompts, I have only spent about $3.
Basic Prompt for Chat or Workbench
This prompt is a very simple way to ask Claude one question about a text that you provide. To use this prompt, put your own text between the text markers. You can use this prompt in the basic Chat interface or the Console Workbench.
USER
Hi Claude – if you did not speak English fluently, which words or phrases in this text do you think you would find difficult to understand?
When you use the Workbench, you will notice a box marked “System Prompt” before the “User Prompt.” You can skip that box for now, but it will come in handy in a few minutes!
The response will appear on the right side of the screen. You might need to adjust your prompt to make it more specific about the format you want to see:
USER
Hi Claude – if you did not speak English fluently, which words or phrases in this text do you think you would find difficult to understand? Please list each word or phrase in your answer on a separate line.
<text> copy and paste here </text>
Once you have the response, you may want to ask Claude to do something else with the same text or to adjust the response. For example, maybe you would rather have the response in the form of a spreadsheet, rather than a list.
Exactly what I wanted! Could you give me that list as a spreadsheet, with each word in Column 1 and its frequency in the text in Column 2?
Before that, though, you need to make sure Claude has a record of the prior responses. This step is one of the big differences between the ChatGPT interface and the ClaudeAI dashboard.
If you are using the Console Workbench and you want to continue asking questions about the same text, you will need to click “Add to Conversation” under the response. The response will now appear on the left, and you will have a new box marked User where you can enter the new prompt. This way, Claude will be able to take prior responses into account.
Guiding Claude with a System Prompt
The System Prompt box lets you give Claude instructions that will be used for any number of User Prompts that you might add below it. In effect, you are “pre-programming” Claude with certain steps, input, or parameters. Many of the examples in the prompt library offer Claude a sort of persona. For example, you could tell Claude to think like an expert editor or something farther afield like a travel agent or even a surly pirate.
In the versions below, I knew I wanted to ask multiple questions about the same text, so I started with a System Prompt that included the text. Then, I only needed to enter my questions in the User Prompt, and I did not always need to use “Add to Conversation.” This method reduces the number of tokens you are submitting each time, which lowers the cost of using the better AI models.
System Prompt
Claude, I would like you to help me analyze the words and phrases that appear in the following text:
<text> copy and paste here </text>
User Prompts
Ask for frequency lists:
USER
Hi Claude – if you did not speak English fluently, which words or phrases in this text do you think you would find difficult to understand? Please list each word or phrase in your answer on a separate line.
The response will appear on the right side of the screen:
ASSISTANT [Claude’s response]
If I were a non-fluent English speaker, I would likely find the following words and phrases challenging:
[list of words and phrases]
These words and phrases are either advanced vocabulary, idiomatic expressions, or specialized terms, which could be challenging for non-fluent English speakers.
That’s a great list! Thank you! Now can you look at the text and make a list of each unique word that appears in the text, from most frequent to least frequent? Please list each word on a separate line and include the number of times it appears in this text.
Great! Now, can you give me a list of words or phrases from this text that appear more frequently in this text than they usually do in other texts in English?
Remember that you entered your text in the System Prompt, so you only need to click “Add to Conversation” if you want Claude to do something based on the prior response.
Ask for the List as a Spreadsheet:
You could also ask for the list to appear as a spreadsheet that you can paste into Excel or Google Sheets.
USER
Hi Claude – if you did not speak English fluently, which words or phrases in this text do you think you would find difficult to understand? Please give me the list as a spreadsheet, with each word in Column 1 and its frequency in the text in Column 2.
<text> copy and paste here </text>
Ask for translation of key words and phrases, i.e., so you can make a glossary:
USER
Hi Claude – if you did not speak English fluently, which words or phrases in this text do you think you would find difficult to understand?
[Add to Conversation]
Now you could ask for translations of challenging words and phrases as a spreadsheet:
Great list! Could you give me that list again with translations into Portuguese? I would like to see the response as a spreadsheet, with the English word or phrase in Column 1 and the translation or paraphrase in Portuguese in Column 2.
OR you could ask for a new version of the text that provides the translations within the text:
Great list! Could you give me a new copy of the text that provides translations of those words for the reader in square brackets? Here is an example showing how I want the translations to appear:
world-readiness [preparação para o mundo]
If you would need to gloss a high number of words and phrases in order to make the text comprehensible, however, it would probably be more valuable to provide a short excerpt or to choose an authentic text on a more familiar topic or aimed at younger readers.
