An exercise conducted at UNSW Business School by Sam Kirshner and Alba Olivares Nadal juggled the idea of workers using ChatGPT as an instrument to write up a cover letter, and explored the intricacies of how it can be applied.

Although on the surface, many would advise against solely relying on an AI tool to create a cover letter. However, having a thorough understanding of the technology can really allow workers to hone a clear and concise cover letter – avoiding the monotonous time-consuming task that it can be.

Cover letters often rarely sway from the generic reiteration of a person explaining their interest in the role, their experience, skills, and other acknowledgments. Therefore it’s not a stretch to say that with some guidance, ChatGPT could achieve this task.

Kirshner and Olivares Nadal explained that being “AI fluent” is a key ingredient to crafting a successful cover letter through the use of ChatGPT. The pair stated that they used an experiment to demonstrate a key pitfall that could arise when going down the AI pathway.

“One group acted as job applicants, and the other as an HR team responsible for hiring. We then handed the job applicants a printout of the job application, which was for a summer intern role as a business analyst. The applicants were given just one minute to use ChatGPT to write a cover letter for the job. Afterwards, we asked them to rate their cover letters out of 10.

“When a few of the applicants read their cover letters aloud, the HR team was impressed. The letters were fluid, articulate, and well-written. The students themselves admitted that the letters were quite good – certainly better than most cover letters from undergrad applicants five or more years ago, and definitely an improvement over the summer internship applications we and our friends submitted 15-plus years ago.”

“However, as we moved on to the second cover letter, something became apparent. It sounded almost identical to the first one – not because it wasn’t fluid, articulate, and well-written, but rather because it was nearly verbatim the same as the first. The same issue arose with the third cover letter, and again with the fourth,” said Kirshner and Olivares Nadal.

VIEW ALL

These identical issues would be a huge problem for applicants, as the HR team looking upon applications would quickly realise the similarities between the differing cover letters and remove them quickly. It’s a scenario that Kirshner and Olivares Nadal believe likely played out for recruiters in the early stages of 2023, when ChatGPT was originally gaining prominence.

As a part of the exercise, the students were also faced with a time limit, which stressed the importance of accurate prompts.

“A key aspect of this exercise was the time limit, which meant that savvier students couldn’t add more personalised details or edit their cover letters beyond using a basic prompt like, ‘Hi ChatGPT, I am applying for the role of Business Analyst at Visionary Consulting; can you write me a cover letter?’ This exercise highlights a broader lesson: many people focus solely on their use of GPT without considering that others are also using ChatGPT simultaneously.

“In the second round, we extended the time limit, giving participants up to 10 minutes to write their cover letters. The goal we set for them was clear: use ChatGPT as much as possible but avoid any dead giveaways that it was used.

“Beyond giving students more time, we also provided a digital copy of the job advertisement so they could copy and paste relevant aspects from the advertisement document directly into ChatGPT. Some students finished within five minutes, while others took the full 10 minutes, using prompts to add more details and personal experiences to their cover letters,” said Kirshner and Olivares Nadal.

With the extra time, the students were asked to read their cover letters aloud, starting with those who finished earlier. Although the letters were improved, on some occasions a phrase would slip up alluding to the issues that can arise when deploying the AI tool.

“For instance, one student’s letter included the phrase, ‘My goal is to revolutionise the fintech industry.’ Beyond being a lofty ambition, the phrase is out of place for an entry-level internship position. When we asked the class what might have given away the use of ChatGPT, no one pointed to this phrase because it blended seamlessly into the letter’s context.

“[We] then asked the students to imagine an interviewer asking about their personal career goals. If they didn’t genuinely respond with the aspiration to ‘revolutionise the fintech industry’, this inconsistency could raise red flags and potentially cost them the job.

“Other students took the time to add more personal details and instructed ChatGPT to adjust the tone of their letters, making them sound less generic and more aligned with their individual voices. These cover letters were noticeably stronger,” said Kirshner and Olivares Nadal.

Overall, the exercise demonstrated the best way to use Chat GPT as a tool to craft a cover letter. By using a more methodical approach, and not completely sprinting through the process, applicants can have some success using Chat GPT.

“The most significant improvement came from students who took a more methodical approach, crafting their cover letters paragraph by paragraph. By working systematically and iteratively with ChatGPT, they were able to adapt the outputs to reflect their own language use and proficiency.

“Ultimately, this approach creates a stronger, more authentic, and more personalised cover letter than those produced by students who relied on a single set of instructions and asked ChatGPT to generate the entire letter in one go, which often resulted in cover letters with lofty, buried goals like ‘revolutionising the fintech industry’.

“Leveraging ChatGPT for crafting cover letters is much like using a high-quality camera: while the tool can capture stunning images, it’s the photographer’s eye that composes the masterpiece. So, the next time you use ChatGPT for a cover letter, remember that while ChatGPT can draft the lines, it’s up to you to provide the genuine substance that truly sets you apart,” said Kirshner and Olivares Nadal.