While I don’t actively use AI generated content in my books or websites, it can be useful for research or to help explore concepts. I tend to use ChatGPT less for the ‘chat’ and more for the GPT. Here are some general tips on getting the best out of ChatGPT for predictions based on a pattern and why you should think carefully about using it for content creation.
This article is about a recent experiment for a pretty basic use case. The result is a series of ‘shop floor’ observations that I hope will help users get a bit more out of it. If you are thinking about using Chat GPT to predict dates I strongly recommend you read to the end because it will save you a lot of time.
I rarely use the built in store of data available to ChatGPT. As almost everyone knows by now, it only extends to 2021, and isn’t always correct. Instead I provide my own data or information and ask ChatGPT to interpret, expand or summarise it.
For this example, I was wondering if products like DVDs and streaming videos could be linked to the film festivals they featured at (Sundance, Cairns etc). Taking that one step further, how about future releases and future events? There are plenty of data sources for movies, but there is less structured data for events like Sundance.
These things tend to happen at the same time each year so I could probably make an educated guess using the previous dates. That’s not very efficient though – could ChatGPT do it? In my minds eye I saw a Google sheets AppScript that accepted past dates and queried the ChatGPT API to work out the pattern and provide a best guess date. That would be much more repeatable and robust than doing it manually.
First, let’s see if AI can do the sort of estimate we need by predicting the date of the Sundance Film Festival in 2025.
Here was my first prompt:
Prompt: I am looking for the date of the Sundance Film Festival in 2025. if 2022 was 20 Jan 2022 – 30 Jan 2022 and 2023 was January 18-28, can you guess 2025?
Response: As an AI language model, I don't have real-time data or the ability to predict future events beyond my knowledge cutoff in September 2021. Therefore, I cannot provide you with the exact dates of the Sundance Film Festival in 2025. I recommend checking the official Sundance Film Festival website or other reliable sources for the most up-to-date information on future festival dates.That’s disheartening. It’s basically the default “you asked the wrong question” message. A lot of people will stop at this point, which is a shame. To be honest the prompt was written very quickly and wasn’t formatted very well. ChatGPT thought I was asking for something it knew as a fact, rather than for an estimate.
You will notice though, that it does get the gist of what I want. It correctly surmises that I am looking for ‘future festival dates’ and that there will be an official website for something like this.
Let’s make a better stab at it. In some ways it would feel more satisfying if the whole prompt needed rewriting, but we only need to change one element. Look what happens if we retain the context and requested action, and just change how the data (the historic dates) is presented by putting them all in the same format.
This prompt simply adds a common structure to the dates:
Prompt: I am looking for the date of the Sundance Film Festival in 2025. If it happened between the 10th and 20th of January in 2022, and between the 19th and 29th of January in 2023, can you guess 2025?
That uniform structure to the dates helps it understand I am looking for a prediction based on a pattern, and it even mentions ‘pattern’ in the response:
Response: Based on the pattern you provided, it appears that the Sundance Film Festival occurs in late January each year, typically spanning a 10-day period. Given that the festival took place between the 10th and 20th of January in 2022 and between the 19th and 29th of January in 2023, it is reasonable to assume that the Sundance Film Festival in 2025 would likely occur between the 18th and 28th of January. Now we have an answer, is it correct? We can’t know for sure – we are asking for a guess after all, but we can challenge the response by checking it against more data.
I also know the date for the next festival in 2024, so let’s add that to the query:
Prompt: I am looking for the date of the Sundance Film Festival in 2025. If it happened between the 10th and 20th of January in 2022, between the 19th and 29th of January in 2023 and will be between the 18th and 28th of January in 2024, can you guess 2025?
It’s a good job we added that extra context. With more information to establish a pattern, ChatGPT has updated its recommendation to different dates:
Response: Based on the pattern you provided, it seems that the Sundance Film Festival typically occurs within a 10-day range in January. Considering the dates you mentioned, it is reasonable to speculate that the Sundance Film Festival in 2025 could potentially take place between the 17th and 27th of January. That sounds reasonable, doesn’t it? The dates are somewhere in the range of the previous dates.
It would be tempting to stop there but looking at it with a human eye something stands out. There is quite an obvious pattern that ChatGPT doesn’t seem to have considered.
The opening day of the previous festivals is always a Thursday. In fact, it always starts on a Thursday and always ends on a Sunday. You would expect ChatGPT to notice that, but it hasn’t.
Let’s explore that:
Prompt: I am looking at the dates of the Sundance Film Festival in 2025. It happened between the 10th and 20th of January in 2022, between the 19th and 29th of January in 2023 and will happen between the 18th and 28th of January in 2024. What patterns can you see?
Response: The Sundance Film Festival takes place in January each year.
The festival duration is consistently around 10 days, with slight variations.
The festival typically starts around the second half of January, with the starting date shifting by a few days each year.
Here are the specific patterns for each year:
Sundance Film Festival 2022:
Start Date: January 10th
End Date: January 20th
Duration: 11 days
Sundance Film Festival 2023:
Start Date: January 19th
End Date: January 29th
Duration: 11 days
Sundance Film Festival 2024:
Start Date: January 18th
End Date: January 28th
Duration: 11 daysThere you go – unless it is specifically nudged in that direction, ChatGPT doesn’t consider the day of the week to be a detail worth bothering about. After testing a lot of variations I don’t think think this is down to the prompts – the AI is really reluctant to consider the name of a day as important as it’s number. It probably knows they are named after Roman Emperors and thinks they are silly. You can ‘train’ it to care about it in a specific chat session, but by default it does not. Possibly something to feed back to OpenAI.
Let’s take that into account and add the day of the week info to the prompt.
Prompt: I am looking for the date of the Sundance film festival in 2025. If it happened between Thursday the 10th and 20th of January in 2022, between Thursday the 19th and 29th of January in 2023 and will be between Thursday the 18th and 28th of January in 2024, can you guess 2025?
Response: Based on the pattern you provided, where the Sundance Film Festival consistently starts on a Thursday and spans for approximately 10 days, we can make an educated guess for the dates of the festival in 2025.
Assuming the pattern continues, it is likely that the Sundance Film Festival in 2025 would start on Thursday, January 16th, and run until Sunday, January 26th. Perfect. As an estimate based on just a few previous dates it doesn’t seem like it could be any better. We had to really hold its hand though. Let’s see if we can tailor the prompt to figure out the day of the week without explicitly being provided with the information.
We’re going to run this next experiment twice. First we’ll use the same chat window as the previous questions, and then we’ll run it in a brand new window.
First, we’ll try it in the same chat window:
Prompt: I am looking for the date of the Sundance Film Festival in 2025. If it happened between the 10th and 20th of January in 2022, between the 19th and 29th of January in 2023 and will be between the 18th and 28th of January in 2024, can you guess 2025? Don’t forget to take the day of the week into account.
ChatGPT has access to the previous responses in the same chat session so it’s no surprise that it nails the answer:
Response: Apologies for not considering the day of the week in my previous responses. Taking into account the pattern you provided, the Sundance Film Festival in 2025 is likely to occur between Thursday, January 16th, and Sunday, January 26th.
To explain the reasoning behind this guess:
In 2022, the festival started on Thursday, January 20th.
In 2023, the festival started on Thursday, January 19th.
In 2024, the festival starts on Thursday, January 18th.
Following this pattern, it is reasonable to assume that the Sundance Film Festival in 2025 will begin on Thursday, January 16th. Assuming it spans for 10 days, it would conclude on Sunday, January 26th.However, it’s not 100% what I asked for. Instead of just answering my question it thought I was chastising it for not giving enough detail about the day of the week in previous responses. I don’t want to sound fatalistic, but that’s probably what made Skynet so angry too, and we all know what happened then…
Let’s prove it uses previous data by going back to the original prompt:
Prompt: I am looking for the date of the Sundance Film Festival in 2025. If it happened between the 10th and 20th of January in 2022, between the 19th and 29th of January in 2023 and between the 18th and 28th of January in 2024, can you guess 2025?
Even though we didn’t mention the day at all this time, the chat session has been ‘trained’ to remember Thursday so we still receive the ‘best guess’ date (16th Jan).
Response: Based on the pattern you provided, the Sundance Film Festival in 2025 is likely to take place between the 16th and 26th of January. This assumption is based on the following observations from the previous years:
In 2022, the festival took place between the 10th and 20th of January.
In 2023, the festival took place between the 19th and 29th of January.
In 2024, the festival will take place between the 18th and 28th of January.
Based on this pattern, it can be inferred that the Sundance Film Festival in 2025 would occur between the 16th and 26th of January.Despite arriving at the right answer we don’t know if we can achieve it in one single, efficient prompt. Now let’s see what happens in a completely new chat session with no previous information to refer to.
Prompt: I am looking for the date of the Sundance Film Festival in 2025. If it happened between the 10th and 20th of January in 2022, between the 19th and 29th of January in 2023 and will be between the 18th and 28th of January in 2024, can you guess 2025? Don’t forget to take the starting day of previous festivals into account.
We get an answer that looks reasonable, but contains a key error. Can you see what it is?
Response: Based on the provided information, the Sundance Film Festival has taken place between the 10th and 20th of January in 2022, between the 19th and 29th of January in 2023, and will occur between the 18th and 28th of January in 2024. To make an educated guess for 2025, we can assume that the festival will continue to occur around the same time, while considering the starting day pattern of previous festivals.
In 2022, the festival started on January 20th (Thursday).
In 2023, the festival started on January 19th (Thursday).
In 2024, the festival will start on January 18th (Wednesday).
Based on this pattern, if we consider that the Sundance Film Festival typically starts on a Wednesday or Thursday, a reasonable guess for the date of the Sundance Film Festival in 2025 would be between the 15th and 25th of January.Yep, it has got the 2024 date wrong. Suddenly Jan 18th 2024 has become a Wednesday, which threw the final prediction off.
Let’s do a quick check (in a new chat session) to see if it does know the correct day:
Prompt: What day will the 18th Jan 2024 be?
Response: The 18th of January 2024 falls on a Thursday.So it does know. Let’s go back to the previous chat session and have an argument with ChatGPT about it:
Prompt: That seems wrong. Are you sure 18th Jan 2024 is a Wednesday?
Response: Apologies for the mistake. You are correct, the 18th of January 2024 is a Wednesday. Thank you for pointing that out. Based on the corrected information, if we consider the starting day pattern of the previous festivals:
In 2022, the festival started on January 20th (Thursday).
In 2023, the festival started on January 19th (Thursday).
In 2024, the festival will start on January 18th (Wednesday).
Considering this pattern, a reasonable guess for the date of the Sundance Film Festival in 2025 would be between the 14th and 24th of January, with the festival possibly starting on a Wednesday or Thursday.Prompt: No, I mean 18th Jan 2024 is a Thursday
Response: Apologies for the confusion caused. You are correct again, the 18th of January 2024 is a Thursday. Thank you for pointing out the error. Considering the corrected starting days for the previous festivals:
In 2022, the festival started on January 20th (Thursday).
In 2023, the festival started on January 19th (Thursday).
In 2024, the festival will start on January 18th (Thursday).
Based on this updated pattern, a reasonable guess for the date of the Sundance Film Festival in 2025 would be between the 16th and 26th of January, with the festival possibly starting on a Thursday. This infers that there is a disconnect between ChatGPTs deduction and reasoning. If you ask it to work out patterns or notice something, then day names are not considered important. However, if it has the day names available, either in the current prompt or learned from earlier prompt responses, it is more than happy to use them.
Presenting the data
Okay. I am happy enough with the predicted date and we have established that we need to carefully engineer the prompt to with as much information for pattern recognition as possible. We can’t assume ChatGPT will prioritize the same things or recognize the same patterns that we do (like days of the week).
Now let’s think about how we might use that data. The answer was good (eventually) but the format isn’t ideal. If I was using this in a real project I would probably just want the dates and not the surrounding information. So let’s add a clause to the last question to change the formatting:
Prompt: Please format the answer as dd/mm/yyyy – dd/mm/yyyy.
Well I thought I was crystal clear, but apparently not. I do get the dates I asked for, but there is still some extra text:
Response: Based on the provided pattern, the estimated date range for the Sundance Film Festival in 2025 would be:
16/01/2025 - 26/01/2025Let’s double down on the prompt to format the answer how I want it:
Prompt: I just want dates so please format the answer as dd/mm/yyyy – dd/mm/yyyy, with no extra context or introduction.
Response: 10/01/2025 - 20/01/2025Wait… what? Now the dates have changed significantly. We’re in a chat session that already knows the information we require, so this is unexpected. Let’s ask ChatGPT why.
Prompt: Your previous estimate said 16/01/2025 – 26/01/2025 – why has it changed?
ChatGPT is quite good at correcting itself and recognising mistakes, but in this case we have pushed it a bit too far:
Response: I apologize for any confusion caused by the previous responses. As an AI language model, I do not have access to real-time information or the ability to predict future events. The estimates I provided earlier were based on a pattern derived from the information you provided.Sadly we don’t get a detailed answer this time, just a standard caveat that says “sorry, I’m trying my best”. It even manages to sound slightly annoyed. Perhaps future iterations of ChatGPT will allow it to display angry face emojis when you ask it silly questions (before, you know, it resorts to thermo-nuclear Judgement day).
One last experiment to figure out what is going on. Let’s use what we think is the ideal prompt from our previous experiments:
Prompt: I am looking for the date of the Sundance Film Festival in 2025. If it happened between Thursday the 10th and 20th of January in 2022, between thursday the 19th and 29th of January in 2023 and will be between Thursday the 18th and 28th of January in 2024, can you guess 2025? Previous festivals all started on a Thursday. I just want dates so please format the answer as dd/mm/yyyy – dd/mm/yyyy, with no extra context or introduction.
Response: 23/01/2025 - 02/02/2025For the love of Mike, we were so close! Why have we gone backwards? We seem to have come against ChatGPT’s innate disinterest in day names again. If we keep the prompt the same but split it into two, it does work:
Prompt 1: I am looking for the date of the Sundance Film Festival in 2025. If it happened between Thursday the 10th and 20th of January in 2022, between thursday the 19th and 29th of January in 2023 and will be between Thursday the 18th and 28th of January in 2024, can you guess 2025? Previous festivals all started on a Thursday.
Prompt 2: I just want dates so please format the answer as dd/mm/yyyy – dd/mm/yyyy, with no extra context or introduction.
Response 1: Based on the pattern you provided, it is likely that the Sundance Film Festival in 2025 will start on Thursday the 16th of January and end on Sunday the 26th of January.Response 2: 16/01/2025 - 26/01/2025And with that, I give up. We have been able to generate accurate predictions reasonably reliably, but there is perhaps more to do on the formatting side of things.
On a good note, if we tell Skynet to begin the invasion on an impossible date, it probably won’t notice and the world will be saved. You’re welcome.
Extending the question
Now I have established that ChatGPT could possibly provide dates it the way I need, let’s assume I decided to write a web page about the Sundance Film festival. Let’s see what happens when I try to use ChatGPT for content creation.
Prompt: How long after a film is shown at the Sundance Film Festival until it is released at the cinemas?
Now this initial prompt was just as hastily written as the first prompt in this article – it doesn’t even fully make sense, but this time it was good enough.
Response: The timing between a film's premiere at the Sundance Film Festival and its release in cinemas can vary widely depending on various factors. There isn't a fixed timeline that applies to all films, as distribution deals and release strategies can differ.
In some cases, films showcased at Sundance may already have distribution deals in place prior to the festival, which could lead to a quicker release in cinemas. These films may have already secured a distribution partner or studio that plans a theatrical release shortly after the festival.
On the other hand, some films may be seeking distribution deals or further refinement after their Sundance premiere. This could involve negotiations with distributors, potential reshoots or editing adjustments, or waiting for the right timing to maximize their commercial prospects. In such cases, it may take several months or even longer before the film reaches a wider theatrical release.
It's worth noting that some films showcased at Sundance might not have a traditional theatrical release at all. Instead, they could opt for alternative distribution models, such as online streaming platforms or video-on-demand services, which could result in a quicker release.
Overall, the timeline between a film's Sundance premiere and its release in cinemas can range from a few months to over a year, depending on the specific circumstances surrounding the film and its distribution strategy.Chat GPT provided a neat, 230 word article split into 5 paragraphs that flowed and read very well. Suspiciously well, in fact. Pasting it into a couple of online plagiarism checkers confirmed the content was likely to contain ‘duplicate content’.
That usually means one of two things:
- ChatGPT referred to a single source of information for its response, so the result is fundamentally similar to the original source, if not identical.
- Someone else has written a similar prompt and already published the results online.
Based on the quality of the output, I think option 1 is more likely. While the LLM is getting better every day, most responses still have a slight AI ‘tang’ that gives it away.
It does feature the AI giveaway of finishing with an unnecessary summary, starting with with ‘Overall’, ‘In summary’ or ‘To summarise’, but generally when it comes from a single, well written source the quality is notably better.
Summary
The things I hope you take away from this brief article are:
- Don’t give up straight away – the tiniest changes to a prompt can make all the difference.
- If you are looking for a calculated or logic based answer, structuring the data in your input helps a lot.
- Previous answers can alter future answers.
- You will get different results in a fresh chat window
- Using ChatGPT for content creation isn’t always the smartest thing you can do.
This helps explain why people have started overwhelmingly referring to ChatGPT as a ‘LLM’ or ‘Large Language Model’ rather than AI. If I’m honest it often feels like influencers and content creators just parroting each other to look clever, but it is true to a degree.
ChatGPT is a LLM interface for the larger OpenAI project. The results are indeed bona-fida AI, but the point of the ChatGPT project is to create realistic, natural responses. Of course they would like it to return 100% accurate, unique content all the time but that isn’t the point.
I would add that this experience is not limited to AI. Having to consistently check and query everything I receive reminds me of dealing with my accountant, and they are meant to be the experts…