Can we use Australian customers’ data to train an AI model?
- Published
- Law stated as at
Key points
- Training or fine-tuning a model on customer records is a use of personal information, so Australian Privacy Principle 6 (‘APP 6’), the rule in the Privacy Act on using and disclosing personal information, applies. If training was not one of the purposes for which the records were collected, it needs the customers’ consent or an exception. The exception that usually matters applies if customers would reasonably expect the training and it is related to the original purpose. If the records include health or other sensitive information, the training must be directly related.
- In the OAIC’s view, adding AI training to a privacy policy after customers’ data was collected will generally not, by itself, change what they reasonably expect, so it will not justify training on that data. Where customers would not clearly expect training, the OAIC advises asking for their consent, or explaining the training clearly and giving them enough time to opt out.
- Data de-identified so that no one is reasonably identifiable is outside the Privacy Act. For example, the medical imaging provider I-MED gave an AI developer patient scans to train a model for diagnostic imaging support, after de-identifying them. In July 2025, the OAIC closed its preliminary inquiries into I-MED without further action, because the data was no longer personal information. It noted that closing the inquiries did not mean it endorsed I-MED’s practices.
- Personal information on public websites is not free to scrape for AI training. By 2021, Clearview AI, a US company, had scraped more than three billion images from the web, including social media, to build a facial recognition search tool. The Australian Information Commissioner found that the facial images were sensitive information collected without consent, and declared that Clearview must stop collecting Australians’ images and destroy those it held. On review, the Administrative Appeals Tribunal agreed that Clearview had collected sensitive information without consent, but was not satisfied that the collection was unfair. The determination stands. In Court Data Australia and Office of the Australian Information Commissioner, decided in 2025, the Administrative Review Tribunal upheld the Commissioner’s finding that Court Data Australia had collected personal information unfairly. That business had copied court lists from a court website into a database that people paid to search. The Tribunal held that information on the public internet is not free for anyone to take and use.
- The rules on training AI may change. From 31 August to 18 September 2026, the government consulted on an exposure draft of the Privacy Amendment (Personal Data Protection) Bill 2026, a draft bill released for public comment before it goes to Parliament. It would replace the separate rules on collection, use and disclosure with a single test: whether the handling is fair and reasonable in the circumstances. As at 4 November 2026, the government is still considering it.
Suppose your product team has three years of chats between your support staff and Australian customers, and wants to use them to fine-tune an AI model. Fine-tuning means training an existing model further on the company’s own examples, so that it answers customers the way your support staff do.
Whether the team can do so depends on the purposes for which the chats were collected, because the Privacy Act, Australia’s main privacy law, ties the use of personal information to those purposes. If training was not one of them, your company needs the customers’ consent or an exception. The exception that usually matters for training applies only if the customers would reasonably expect it and it is related to the purpose for which the chats were collected. Where the chats contain health or other sensitive information, the training must be directly related to the purpose of collection, not merely related.1
Many companies assume that they can solve this problem by adding AI training to their privacy policy. In the view of the Office of the Australian Information Commissioner (‘OAIC’), the federal regulator that enforces the Privacy Act, that will generally not work for data a company already holds. The reason is that what customers reasonably expect depends largely on what they were told when the data was collected, not on what a policy says later.2
Which companies does the Privacy Act apply to?
The Privacy Act applies to Australian companies, and to overseas companies that carry on business in Australia, unless they are exempt as small businesses.3 If your company is based outside Australia, a separate article on overseas companies explains when the Privacy Act applies to it.
A business with an annual turnover of $3 million or less is generally exempt as a small business operator, and the rules discussed in this article do not apply to it. The exemption does not cover every small business, however. Whatever its turnover, a business is not exempt if, for example, it provides a health service and holds health information, or if it trades in personal information, such as by selling customer lists. An exempt small business can also choose to opt in, so that the Privacy Act binds it as it binds a larger business.4
Is customer data still personal information once it is in a training set?
Customer data usually remains personal information when it goes into a training set, the collection of examples from which a model learns. Personal information is information or an opinion about an identified individual, or about an individual who is reasonably identifiable from it, whether it is true or not.5 So a wrong note in a customer’s account, such as a record that they missed a payment when in fact they paid on time, is still personal information about them.
Support chats, account histories and uploaded photos are personal information if a person can be identified from them. The OAIC says that what a model produces about an identified or reasonably identifiable person is personal information too. That includes guesses the model makes about a person, such as a customer’s likely income. It also includes false statements that the model presents as fact, known as ‘hallucinations’, and realistic fake images, video or audio of a person, known as ‘deepfakes’.6
What counts as sensitive information?
Sensitive information is a kind of personal information that the Privacy Act protects more strictly than the rest. It includes health information (including about illness or disability), and information about racial or ethnic origin, religious beliefs or sexual orientation. It also includes biometric information, such as an image of a face or a fingerprint, that is to be used to verify or identify a person automatically. Biometric templates, the digital records of a person’s features made for that purpose, are sensitive information too.7 A support chat in which a customer describes a medical condition contains sensitive information.
Three stricter rules for sensitive information matter for training:
- A company generally needs consent to collect sensitive information.8
- Without consent, a company can usually use sensitive information for a new purpose only if the person would reasonably expect it and the new purpose is directly related to the original one, not merely related.9
- A company can pass personal information that is not sensitive to a related company, meaning another company in the same corporate group, such as its parent. Sensitive information can be passed on only with consent or under an exception.10
Does de-identified data fall outside the Privacy Act?
Data that has been de-identified, so that it is no longer about an identified or reasonably identifiable person, falls outside the Privacy Act. Because it is no longer personal information, it cannot be sensitive information either.11 De-identifying data is therefore one way to take it outside the rules above. But the OAIC treats the process of de-identifying as itself a use of the personal information. It also warns that whether de-identification works depends on the context, and that it may be difficult to achieve.12
Free-text chats show why de-identification can be difficult. In a table of customer records, names and contact details sit in their own columns and can be deleted. In a chat, customers mention their name, address, employer or an unusual detail anywhere in the text, in their own words, so a tool that removes names can miss them. And the details that remain, such as a rare condition and a suburb, can single someone out when combined with other information, such as the company’s order records. So removing names is not enough: the chats are de-identified only when no one is reasonably identifiable from them.13
Meeting that standard can take several steps, as the OAIC’s preliminary inquiries into I-MED, a provider of medical imaging such as X-rays and scans, show. Between April 2020 and January 2022, I-MED gave patient scans and reports to Annalise.ai, then a business owned jointly by I-MED and another company, Harrison.ai. Annalise.ai used them to develop and train an AI model to improve diagnostic imaging support, that is, help in reading medical scans. Patients were not told and did not consent.14
Before sharing the data, I-MED took six steps. It separated the data from the rest of its records and scanned the records with text recognition software. It hashed unique identifiers, names, addresses and phone numbers, which means it replaced them with scrambled codes, and it shifted dates within a set range. It grouped some fields into large categories so that unusual cases would not stand out, and it blacked out text in and near the borders of the scans.15
Technical steps were not the only safeguard. I-MED’s contract also barred Annalise.ai from doing anything that would make the data reasonably identifiable and from disclosing it, and required it to store the data securely.16
Together, these steps did not entirely remove the risk that a patient could be identified again. But they reduced that risk to a sufficiently low level, and they were supported by sound data governance, meaning clear internal rules and controls on who could use the data and how. The Privacy Commissioner was therefore satisfied that the data shared was no longer personal information. For that reason, and because the few accidental disclosures were minor and had been dealt with, the Commissioner closed the inquiries with no further action. Even so, the OAIC’s Report into Preliminary Inquiries of I-MED, published in July 2025, says that closing the inquiries should not be taken as approval of I-MED’s practices.17
When can a company use customer data for a new purpose?
A company can use customer data for a new purpose, such as training a model, only with the customers’ consent or under an exception. The rule is APP 6, one of the 13 Australian Privacy Principles in the Privacy Act that set out how companies must handle personal information. Under APP 6, information collected for one purpose, called the primary purpose, must not be used for another purpose without the person’s consent, unless an exception applies.18 Most of the exceptions deal with particular situations, such as a use required by an Australian law, and rarely fit a company’s own AI training. One exception can matter for health information. An organisation may use health information without consent for research relevant to public health or public safety, if seeking consent is impracticable and the research follows guidelines approved under the Privacy Act.19
For most training, the exception that matters has two parts, and both must be met: the customer must reasonably expect the use, and the new purpose must be related to the original purpose. For sensitive information, the new purpose must be directly related.20
The OAIC’s guidelines explain what the second part requires. A related purpose is connected to or associated with the original purpose, and the link must be more than tenuous, that is, more than weak or remote. A directly related purpose, which sensitive information requires, is closely associated with the original purpose, although the new use need not be strictly necessary to achieve it. In one OAIC example, a health provider who cannot treat a patient tells another provider at the same clinic that the patient needs treatment. That disclosure is directly related to the purpose for which the information was collected.21
The first part, reasonable expectation, is where most plans to train on existing data fail. The test is objective: it asks what a reasonable person, properly informed, would expect, not what one customer in fact expected. In practice, what the company told customers when it collected their data carries particular weight.22 In a hypothetical OAIC example, a social media company that has collected users’ posts since 2010 to run its platform wants, in 2024, to train a large language model on them to license to others. The OAIC says that use would not be permitted, because users would not have expected it and the link to the original purpose of running the platform is too weak.23
Two shortcuts that companies often rely on will not usually make training on data they already hold lawful. One is a broad consent, such as agreement that data will be handled ‘in accordance with our privacy policy’, which is too general to count as consent to training. The other is a privacy policy or notice updated after the data was collected, which does not change what customers expected when they handed the data over.24
Data collected after such an update is in a better position, because customers are told about training before they hand it over. In another hypothetical example, an AI developer tells users in advance that the questions they type into its public chatbot, called prompts, will be used to retrain later versions of the chatbot. The OAIC considers that use likely to be expected.25
That leaves the two routes that matter for most training. Consent and the exception are separate, so a company needs one or the other, not both.26 Where it is unclear whether customers would expect the training, the OAIC advises seeking their consent, offering a clear opt-out, or both. An opt-out is a simple way for customers to say no, and it is clear only if customers have enough information and time to use it. The OAIC presents these steps as a way to reduce the risk of action by the regulator.27
An opt-out is not a third route. It helps because, in the OAIC’s view, customers who were clearly told about the training and did not opt out may be taken to have consented by implication. The OAIC says it will be appropriate to infer consent in this way only in limited circumstances, such as where the opt-out was clearly presented, easy to use and not bundled with other purposes. For sensitive information, the OAIC says a company should generally seek express consent.28
Can a parent company train on the data?
A parent company is generally not free to train on data that its Australian subsidiary collected. Where the parent is itself bound by the Privacy Act, two rules set its limits. The first is about passing the data to the parent.
For information that is not sensitive, the Privacy Act makes an exception within a group. Disclosing it to a related company, such as a parent, is not an interference with privacy, so no consent or exception is needed.29 Sensitive information has no such exception. It may still be passed on for the original purpose, but passing it on so that the parent can train a model is a disclosure for a new purpose. That needs the customers’ consent, or a disclosure they would reasonably expect that is directly related to the original purpose, which is the stricter test that applies to sensitive information.30
The second rule is about the parent’s own use. A related company bound by the Privacy Act is treated as having collected the information for the Australian company’s original purpose. The parent therefore stands in exactly the same position as the Australian company. It may use the data for that original purpose. To train a model on it for a new purpose, it needs the customers’ consent, or a use they would reasonably expect that is related to the original purpose and, for sensitive information, directly related.31
In the OAIC’s view, if the parent is overseas, sending the data to it is also a cross-border disclosure, which APP 8 regulates.32 Before sending the data, the Australian company must take reasonable steps to ensure that the overseas company does not breach the Australian Privacy Principles. Taking those steps does not end the Australian company’s responsibility. If the overseas company then does something that would breach the principles, the Privacy Act treats it as the Australian company’s own breach, unless the overseas company is itself bound by the Privacy Act. So a parent that is not bound by the Privacy Act is not free to train on the data either. If it does something with the data that would breach the principles, the Australian company answers for it.33
Two narrow exceptions remove both the duty to take reasonable steps and this responsibility. One applies if the Australian company reasonably believes that the overseas company is bound by a law that protects the information in a substantially similar way and that customers can enforce. Another applies if a customer consents after being told expressly that the Australian company will then not have to take those steps.34
The rules on sending data overseas are explained in detail in a separate article on sending customer data to an overseas head office.
We are fine-tuning an AI vendor’s model. Does that change anything?
Most companies that train AI on customer data do not build a model from scratch. They fine-tune or customise a model supplied by an AI vendor, often on the vendor’s own platform. The same rules apply, and there is one more risk.
The OAIC’s guidance treats a company that fine-tunes someone else’s model as a developer, so the guidance for developers discussed above applies to it.35 In the OAIC’s view, putting customer data into a vendor’s AI product is a use of the data if the data stays within the company’s control. That is the case, for example, where safeguards ensure that the data is not passed outside the company, including to the vendor. It is a disclosure if the data is made available to people outside the company, such as the vendor, and the company no longer has effective control over it. The data has then, in effect, been given to someone else. Either way, the company needs consent, or a use or disclosure customers would reasonably expect that is related to the original purpose.36
The extra risk is in the vendor’s terms, because some products’ terms or settings let the vendor collect what users put in, such as prompts and uploaded data, to train its own models further. Customer data put into such a product leaves the company’s control, and could surface in answers given to other users.37 To manage that risk, the OAIC recommends checking whether a product suits its intended use, reviewing its terms and settings carefully, and turning off features that would disclose personal information. It also recommends a privacy impact assessment, a written review of how a project will handle personal information and how to reduce the risks to privacy. As a matter of best practice, the OAIC advises against entering personal information, particularly sensitive information, into publicly available AI chatbots.38
The OAIC’s recommendations are a minimum. Practical steps that go further include choosing a business plan whose terms say the vendor will not train on your data. The contract can record that promise, with limits on retention, storage location and deletion. Personal details can also be removed or masked before any data is uploaded.
What about data scraped from public websites?
Personal information on public websites is not free to use. Gathering it from the web with software that copies content automatically, known as scraping, is a collection of personal information, so the rules on collecting personal information apply. The collection must be reasonably necessary for the company’s functions or activities, and done by lawful and fair means.39 The information must also be collected from the person it is about, unless that is unreasonable or impracticable.40
Scraping takes information from websites, not from the people it is about, so a company that scrapes must be able to justify not collecting from those people directly. The OAIC’s guidelines list relevant factors: whether people would expect collection from another source, how sensitive the information is, the privacy risk, and the time and cost of collecting directly. Inconvenience or cost alone is not enough.41
Collecting sensitive information, such as biometric information, also needs consent, even from a publicly available website, unless an exception applies.42 The OAIC adds that public availability does not make data lawful to use for training. It also says that building a dataset by scraping is generally a covert way of collecting, meaning one that is hidden from the people the information is about.43
Clearview AI’s case, Commissioner Initiated Investigation into Clearview AI, Inc (Privacy) (‘Clearview’), decided in 2021 by the Australian Information Commissioner, who heads the OAIC, shows the cost of breaking the collection rules. Clearview, a company incorporated in the United States, offers a facial recognition search tool. It told the Commissioner that it supplies the tool only to government customers, for law enforcement and national security. A user uploads a photo of a face, and the tool searches its database for likely matches. When the Commissioner decided the case, the database held more than three billion images.44 Clearview’s web crawler gathered those images from publicly available websites, including social media, and turned each face into a mathematical vector, a string of numbers that describes the features of the face.45
Clearview argued that it collected only publicly available images and handled no personal information. The Commissioner rejected that argument. The Commissioner found that the facial images and vectors were personal information, and sensitive information as well, so collecting them without consent breached the rule requiring consent to collect sensitive information.46
The Commissioner also found the collection unfair, although the tribunal that later reviewed the determination was not satisfied that it was. In the Commissioner’s view, the collection was covert: Clearview did not tell the people in the images, many of whom were unlikely to have posted the images themselves. The terms of the social media sites it scraped also prohibited scraping. The Commissioner accepted that covert collection is not always unfair. What made this collection unfair, in the Commissioner’s view, was that it was also unreasonably intrusive.47
In the Commissioner’s view, three features made the collection unreasonably intrusive. First, the images were biometric: they could be used to identify a person by their face. Second, the database covered millions of people, including children and victims of crime, and only a fraction of them would ever be connected with a law enforcement investigation. Third, Clearview collected the images for its own commercial ends, including training and improving its algorithm.48
The Commissioner also found that Clearview had breached the principles on telling people about collection, on keeping information accurate and on managing privacy within the company. As a result, the Commissioner made a determination under the Privacy Act and declared that Clearview must stop collecting images from people in Australia and must destroy what it had collected from them within 90 days.49
Clearview asked the Administrative Appeals Tribunal (now the Administrative Review Tribunal), a tribunal that could decide government decisions again on their merits, to review the determination. In reasons given in May 2023, the Tribunal agreed that the Privacy Act applied to Clearview. It also found that Clearview had breached the rule requiring consent to collect sensitive information, and the principle on managing privacy within the company.50
But the Tribunal was not satisfied that three other breaches found by the Commissioner were made out. They were unfair collection, failing to tell people about the collection, and failing to take reasonable steps to keep the information accurate. On fairness, the Tribunal reasoned that people who put an image on the internet without restricting access have chosen to make it public, so collecting it with a web crawler is not unfair. It added that the result might be different if a person had posted an image on the understanding that limits would apply to its use, and the collection broke those limits.51 The Commissioner recorded these conclusions in a 2024 determination, ‘AHM’ and JFA (Aust) Pty Ltd t/a Court Data Australia (Privacy) (‘Court Data Australia’).52
The Tribunal did not make a formal decision: it left the question of what orders to make for a later hearing.53 The OAIC says that Clearview withdrew its application in August 2023, before that hearing, so the Clearview determination remains in force.54
In early 2024, media reports suggested that Clearview was still collecting Australians’ images. The OAIC did not decide the case again. Its Privacy Commissioner, Carly Kind, considered only whether to take further action against Clearview, and in August 2024 decided that further action was not warranted at that time. The Commissioner noted that Clearview had faced investigations by regulators in at least three other jurisdictions, and said the determination still stands.55
The collection rules reach more than faces. In Court Data Australia, the Australian Information Commissioner considered the practices of JFA (Aust) Pty Ltd, which trades as Court Data Australia. The company copied the daily lists of cases that Australian courts publish, including those on the Western Australian courts’ website, into a database. Customers could search the database by name, and paid to see the details.56
The person who complained to the OAIC, identified in the determination only as ‘AHM’, had faced six criminal charges in Western Australia, which were later dismissed. The database held 52 entries copied from the courts’ daily lists that showed the complainant’s name and those charges, but nothing in it showed that the charges had been dismissed. Through an advocate acting on their behalf, the complainant asked the company to remove the entries, and it refused.57
The Commissioner accepted that it was impracticable for the company to collect the information from each person directly. But the Commissioner found the collection unfair, and distinguished the Tribunal’s reasoning in Clearview’s case. The complainant had not chosen to publish the information: it appeared in the court lists because courts sit in public. The website’s terms of use prohibited reusing its material for any commercial purpose without permission. And the complainant did not know of the collection and could not reasonably have expected it.58
The Commissioner also found that, although the company could not practicably notify each person whose name it collected, it should have published a complete notice of its collection on its own website.59 The Commissioner declared that the company must stop collecting personal information from that website in breach of its terms and without the knowledge of the person concerned. It also had to destroy, within 30 days, what it had collected in that way.60
Court Data Australia sought review, and in May 2025 the Administrative Review Tribunal affirmed the determination. It held that collecting by automated means is not unfair in itself. But it found that the complainant could not reasonably have expected their information to be taken from daily court lists into a permanent commercial database. Court lists serve open justice and are short-lived, while the database kept listings indefinitely and invited its users to draw adverse inferences. Putting personal information on the public internet, it said, does not make it ‘a free for all’.61
In practice, a company building a training set from the web should take four steps, which follow from the rules and the two cases above.
- Screen out Australians’ personal information that it does not need.
- Treat recognisable faces as sensitive information that needs consent. In Clearview’s case, the Commissioner found, and the Tribunal agreed, that the facial images and the vectors made from them were sensitive information. The OAIC adds that an image of a person is sensitive information where it implies, for example, the person’s race or a medical condition, or where it will be used to identify people automatically.62
- Check each website’s terms of use, and do not collect in breach of them. In Court Data Australia, the website’s terms barred commercial reuse, and both the Commissioner and the Tribunal treated that as part of what made the collection unfair.63
- If it cannot tell each person about the collection, explain the collection on its own website. In Court Data Australia, both the Commissioner and the Tribunal found that the company should have done so.64
Summary: which data can a company train a model on?
The table brings together, for each kind of data, the main rules explained above.65
| Your data | Can you train a model on it? |
|---|---|
| Collected to run your service, without telling customers about training | Only with consent, or if customers would reasonably expect training and it is related to why you collected the data. Updating your policy now will not usually be enough |
| Collected after telling customers it may be used for training | Yes, if training was one of the purposes for which the data was collected. If customers were only told it may later be used for training, they are more likely to expect it, but training must still be related to why you collected the data |
| Includes health or other sensitive information | Only with consent, or if customers would reasonably expect training and it is directly related to why you collected the data |
| De-identified, so that no one is reasonably identifiable | Yes. It is no longer personal information, though de-identifying it is itself a use of the personal information |
| Scraped from public websites | Only if collecting it is reasonably necessary and done by lawful and fair means. Sensitive information, such as faces used for recognition, generally needs consent |
What happens if a company breaks these rules?
Breaking any of the rules in the table, including the collection rules that Clearview breached, is an ‘interference with privacy’ under the Privacy Act, and can lead to a civil penalty. The determination against Clearview imposed no fine, because only a court can order one. For a serious interference with privacy, the Federal Court or the Federal Circuit and Family Court of Australia can order a civil penalty if the Commissioner applies for one. For a company, the maximum is $50 million or, if greater, three times the benefit from the breach. If the court cannot value that benefit, the comparison is with 30% of the group’s adjusted turnover (broadly, its sales connected with Australia, not counting GST) over at least 12 months.66
Any interference with privacy, serious or not, can also attract a lower civil penalty: up to 2,000 penalty units for an individual and five times that, 10,000 penalty units, for a company. A penalty unit is a dollar amount set under federal law and indexed every three years. For a breach on or after 1 July 2026, a penalty unit is $364, which makes the company maximum $3.64 million. If the Commissioner applies for the higher penalty and the court is not satisfied that the interference was serious, it can order the lower one instead.67
Worked example: what should a company do before training on customer data?
The example below is hypothetical. It shows how the rules explained above apply to one company, step by step.
Say a smart-home brand headquartered in Shenzhen carries on business in Australia, selling cameras and door locks to Australians through its own app. Its sales to Australians alone are about $9 million a year, so the small business exemption is not available.68 Since March 2023, it has kept 400,000 support chats with Australian customers. They were collected under a collection notice, the short statement shown to customers when their information is collected, which said the chats would be used ‘to provide support and improve our services’. In November 2026, its product team wants to fine-tune a support chatbot on them, but some chats contain health information, because customers mention a disability or medical condition when asking about accessibility settings. The example assumes that the brand collected those chats lawfully; if it did not, that problem comes first.
The Shenzhen brand should take six steps.
- Set aside the chats that mention health. They contain sensitive information, so the stricter test applies: the brand can train on them only with the customers’ consent, or if customers would reasonably expect the training and it is directly related to providing support. In practice, the simplest course is to leave these chats out of the training set.69
- Remove what the model does not need from the other chats, such as names, contact details and order numbers. This lowers the risk, but it may not de-identify the chats, because customers mention identifying details in their own words. So the chats will usually still be personal information, and the next step still applies.70
- Decide the legal basis for training on the other chats. Training a support chatbot on support chats is related to providing support, so the related-purpose part of the exception is likely to be met. The reasonable-expectation part is doubtful. The brand’s own collection notice, under which the chats were collected, said only that they would be used ‘to provide support and improve our services’ and did not mention AI. In the OAIC’s view, a notice or policy changed now will generally not make customers expect training on chats already sent. The cautious course is therefore to ask for consent, or to give Australian customers a clear notice and a real opt-out, with enough time to use it before training starts. As explained above, an opt-out works only as implied consent, so the notice must explain the training clearly and the opt-out must be easy to use.71
- Tell customers about training before new chats begin. A collection notice must, so far as is reasonable, state the purposes of collection.72 If the notice says that chats may be used to train the support chatbot, customers who chat afterwards are more likely to expect it, as in the OAIC’s chatbot example.73
- Check the data, and the vendor if one is used. The brand must take reasonable steps to ensure that the chats it trains on are accurate, up to date, complete and relevant to the purpose.74 Many companies fine-tune a model supplied by an AI vendor, on the vendor’s own platform. If the brand does, it should first read the vendor’s terms on what the vendor may do with the chats, because some vendors’ terms let them keep what users upload to train their own models, as explained above.
- Delete what is no longer needed. Once the brand no longer needs the chats for any permitted purpose, it must take reasonable steps to destroy or de-identify them, unless a law requires them to be kept. The Privacy Act sets no fixed time for doing so: the duty arises when the need ends.75
Must the privacy policy describe the chatbot’s decisions from 10 December 2026?
From 10 December 2026, the brand’s privacy policy may have to describe the kinds of decisions that the chatbot makes about customers, or substantially and directly helps to make. It may also have to describe the kinds of personal information the chatbot uses for those decisions. The duty applies only where the chatbot uses customers’ personal information to make those decisions, and only to decisions that could reasonably be expected to significantly affect customers’ rights or interests. A chatbot that decides whether a customer gets a refund or a replacement could make such a decision, because the Privacy Act gives decisions affecting a person’s rights under a contract as an example.76 That duty is explained in a separate article on automated decisions.
What is not settled (as at 4 November 2026)
- The test for handling personal information may change. From 31 August to 18 September 2026, the Australian Government, through the Attorney-General’s Department, consulted on an exposure draft of the Privacy Amendment (Personal Data Protection) Bill 2026, a draft released for public comment before a bill goes to Parliament. The draft would replace APP 3 and APP 4, the rules on collecting personal information, and APP 6, the rule on using and disclosing it, with one test. Under that test, any handling of personal information must be fair and reasonable in the circumstances. Whether it is would be judged against factors including what people reasonably expect, how open the company is about the handling, whether people have a genuine choice, and whether the handling goes further than its purpose needs. A purpose arising after collection, such as training, would be less likely to pass if it is outside reasonable expectations, lacks transparency or offers no genuine choice. A clear opt-out could help, because the consultation paper lists opt-out mechanisms as one way to support genuine choice, but no single factor would decide the question. The draft remains under government consideration.77
- The meaning of de-identified data may change. The draft treats de-identification as ‘not a fixed state’: whether data is de-identified would depend on the circumstances, which can change. In other words, data that counts as de-identified in one setting might not count as de-identified in another.78
- People may gain a right to have their information erased. The draft would give people a right to ask large digital platforms to erase their personal information. By contrast, the Privacy Act gives no general right of erasure now. People can ask to access and correct their information, and a company must destroy or de-identify information it no longer needs.79
- No court has yet decided how the current rules apply to AI training, so far as the research for this article found. Until one does, the main guide is what the OAIC has published, such as its guidance on AI and its Australian Privacy Principles Guidelines, which explain each principle. These documents show how the regulator reads the Privacy Act, but they are not the law itself.80
Questions
- Our privacy policy says we use data to improve our services. Does that cover AI training?
- A clause about improving services will probably not cover AI training on its own. The OAIC says a broad consent to handling under the privacy policy is not enough, and whether customers would reasonably expect training is judged objectively, with weight on what they were told when the data was collected.
- Can our overseas parent train a group model on our Australian customers’ data?
- An overseas parent cannot freely train a group model on the data, whether or not it is itself bound by the Privacy Act. The Privacy Act lets a company pass personal information that is not sensitive to a related company, but a parent bound by the Privacy Act takes on the Australian company’s original purpose, so the same test for training applies. Sensitive information needs consent or an exception before it is passed on. In the OAIC’s view, sending data offshore also brings in APP 8, the rule on sending personal information overseas, and if a parent that is not bound by the Privacy Act does something that would breach the Australian Privacy Principles, the Australian company is treated as having done it.
- Do we need consent if we de-identify the data first?
- Consent is not needed for data that is truly de-identified, because it is no longer personal information. The OAIC treats de-identifying as itself a use of the personal information, and warns that it depends on context and can be hard to achieve.
This article is general information about Australian law as at the date shown above. It is not legal advice and does not take account of your circumstances. Reading it does not create a lawyer–client relationship with SQ Legal. Obtain advice on your own matter before acting. See the terms of use.
Related
- Does the Australian Privacy Act apply to a company based outside Australia?
- What must a privacy policy say about automated decisions from 10 December 2026?
- Can we send Australian customers’ personal information to our head office overseas?
More on data and privacy, or contact SQ Legal about a matter.
Anna Zheng, ‘Can we use Australian customers’ data to train an AI model?’, SQ Legal (Web Page, 16 November 2026) <https://sqlegal.com.au/insights/ai-training-personal-information-australia/>.
You may quote short extracts with attribution and a link to this page. To republish the article in full, write to [email protected]Email[email protected]Open in your default email appGmailOutlook (personal)Outlook (work or school)Or copy the address:[email protected].
Footnotes
-
Privacy Act 1988 (Cth) sch 1 cls 6.1, 6.2(a) (‘Privacy Act’). ↩
-
Office of the Australian Information Commissioner, Guidance on Privacy and Developing and Training Generative AI Models (Guidance, 21 October 2024) (‘Training Generative AI Models’). ↩
-
Privacy Act (n 1) s 5B(1A), (3). ↩
-
Ibid ss 6C(1) (definition of ‘organisation’), 6D(1), (4), 6EA. ↩
-
Ibid s 6(1) (definition of ‘personal information’). ↩
-
Training Generative AI Models (n 2). ↩
-
Privacy Act (n 1) ss 6(1) (definition of ‘sensitive information’), 6FA(a)(i). ↩
-
Ibid sch 1 cl 3.3. ↩
-
Ibid sch 1 cl 6.2(a)(i). ↩
-
Ibid s 13B(1)(b), sch 1 cl 6.1. ↩
-
Ibid s 6(1) (definitions of ‘de-identified’ and ‘sensitive information’). ↩
-
Training Generative AI Models (n 2). ↩
-
Privacy Act (n 1) s 6(1) (definition of ‘de-identified’). ↩
-
Office of the Australian Information Commissioner, Report into Preliminary Inquiries of I-MED (Report, 31 July 2025) [1.3], [3.5], [6.2]. ↩
-
Ibid [5.1]. ↩
-
Ibid [5.2]. ↩
-
Ibid [1.5], [7.1]–[7.2]. ↩
-
Privacy Act (n 1) sch 1 cl 6.1. ↩
-
Privacy Act (n 1) s 16B(3), sch 1 cl 6.2(b)–(e). ↩
-
Ibid sch 1 cl 6.2(a). ↩
-
Office of the Australian Information Commissioner, ‘Chapter 6: APP 6 Use or Disclosure of Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 22 July 2019) [6.24], [6.26]–[6.27]. ↩
-
Ibid [6.20]; Training Generative AI Models (n 2). ↩
-
Training Generative AI Models (n 2). ↩
-
Ibid. ↩
-
Ibid. ↩
-
Privacy Act (n 1) sch 1 cl 6.1(a)–(b). ↩
-
Training Generative AI Models (n 2). ↩
-
Office of the Australian Information Commissioner, ‘Chapter B: Key Concepts’ in Australian Privacy Principles Guidelines (Guidelines, 21 December 2022) [B.43]–[B.44]. ↩
-
Privacy Act (n 1) s 13B(1)(b). ↩
-
Ibid s 13B(1)(b), sch 1 cls 6.1, 6.2(a)(i). ↩
-
Ibid sch 1 cls 6.2(a), 6.6. ↩
-
Office of the Australian Information Commissioner, ‘Chapter 8: APP 8 Cross-Border Disclosure of Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 3 October 2025) [8.6]; Privacy Act (n 1) sch 1 cl 8.1. ↩
-
Privacy Act (n 1) s 16C, sch 1 cl 8.1; Office of the Australian Information Commissioner, ‘Chapter 8: APP 8 Cross-Border Disclosure of Personal Information’ (n 32) [8.62]. ↩
-
Privacy Act (n 1) s 16C(1)(b), sch 1 cl 8.2(a)–(b). ↩
-
Training Generative AI Models (n 2). ↩
-
Office of the Australian Information Commissioner, Guidance on Privacy and the Use of Commercially Available AI Products (Guidance, 21 October 2024) (‘Commercially Available AI Products’); Privacy Act (n 1) sch 1 cls 6.1, 6.2(a). ↩
-
Commercially Available AI Products (n 36). ↩
-
Ibid. ↩
-
Privacy Act (n 1) sch 1 cls 3.2, 3.5. ↩
-
Ibid sch 1 cl 3.6. ↩
-
Office of the Australian Information Commissioner, ‘Chapter 3: APP 3 Collection of Solicited Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 13 May 2026) [3.90]. ↩
-
Privacy Act (n 1) sch 1 cl 3.3; Training Generative AI Models (n 2). ↩
-
Training Generative AI Models (n 2). ↩
-
Commissioner Initiated Investigation into Clearview AI, Inc (Privacy) [2021] AICmr 54, [3]–[4], [6], [32] (‘Clearview’). ↩
-
Ibid [5]. ↩
-
Ibid [98], [103], [110], [145], [167]. ↩
-
Ibid [171]–[173], [177], [179]–[180]. ↩
-
Ibid [178]–[180]. ↩
-
Ibid [1]–[2]. ↩
-
Clearview AI Inc and Australian Information Commissioner [2023] AATA 1069, [124]–[131], [195]–[199] (‘Clearview Review’). ↩
-
Ibid [132]–[134], [140], [147]. ↩
-
‘AHM’ and JFA (Aust) Pty Ltd t/a Court Data Australia (Privacy) [2024] AICmr 29, [63]–[64], [98], [139] (‘Court Data Australia’). ↩
-
Clearview Review (n 50) [201]. ↩
-
Office of the Australian Information Commissioner, ‘Statement on Clearview AI’ (Media Release, 21 August 2024). ↩
-
Ibid. ↩
-
Court Data Australia (n 52) [3]–[7]. ↩
-
Ibid [9]–[11], [35], [108]. ↩
-
Ibid [65], [68], [71], [79], [84], [86]. ↩
-
Ibid [96], [101]–[102]. ↩
-
Ibid [150]. ↩
-
Court Data Australia and Office of the Australian Information Commissioner [2025] ARTA 876, [32], [35]–[37], [41]–[42], [50], [61] (‘Court Data Australia Review’). ↩
-
Clearview (n 44) [145]; Clearview Review (n 50) [126]–[129]; Training Generative AI Models (n 2). ↩
-
Court Data Australia (n 52) [71], [86]; Court Data Australia Review (n 61) [40]. ↩
-
Court Data Australia (n 52) [101]–[102]; Court Data Australia Review (n 61) [49]–[50]. ↩
-
Privacy Act (n 1) s 6(1) (definition of ‘de-identified’), sch 1 cls 3.2, 3.3, 3.5, 6.1, 6.2(a); Training Generative AI Models (n 2). ↩
-
Privacy Act (n 1) ss 13(1)(a), 13G(1), (3), (5), (7), 80U(1)–(3); Clearview (n 44) [2]. ↩
-
Privacy Act (n 1) ss 13H, 13J; Regulatory Powers (Standard Provisions) Act 2014 (Cth) s 82(5); Crimes Act 1914 (Cth) s 4AA(1), (3); Crimes (Amount of a Penalty Unit) Instrument 2026 (Cth) s 5; ‘Fines and Penalties’, Australian Securities and Investments Commission (Web Page) <https://www.asic.gov.au/about-asic/asic-investigations-and-enforcement/fines-and-penalties>. ↩
-
Privacy Act (n 1) s 6D(1). ↩
-
Ibid sch 1 cl 6.2(a)(i). ↩
-
Ibid s 6(1) (definition of ‘de-identified’); Training Generative AI Models (n 2). ↩
-
Training Generative AI Models (n 2); Office of the Australian Information Commissioner, ‘Chapter B: Key Concepts’ (n 28) [B.43]. ↩
-
Privacy Act (n 1) sch 1 cls 5.1, 5.2(d). ↩
-
Training Generative AI Models (n 2). ↩
-
Privacy Act (n 1) sch 1 cl 10.2. ↩
-
Ibid sch 1 cl 11.2. ↩
-
Privacy and Other Legislation Amendment Act 2024 (Cth) s 2(1) table item 7, sch 1 item 88, inserting Privacy Act (n 1) sch 1 cls 1.7–1.9. ↩
-
‘Privacy Reform: Consultation on Exposure Draft Legislation’, Attorney-General’s Department (Web Page, 31 August 2026) <https://consultations.ag.gov.au/rights-and-protections/privacy-reform/>; Attorney-General’s Department, Privacy Reform (Consultation Paper, August 2026) 1, 10–11, 14. ↩
-
Privacy Reform (n 77) 4. ↩
-
Ibid 2; Privacy Act (n 1) sch 1 cls 11.2, 12.1, 13.1. ↩
-
Clearview (n 44) [29]. ↩
Sources
- Privacy Act 1988 (Cth)
- Regulatory Powers (Standard Provisions) Act 2014 (Cth)
- Crimes Act 1914 (Cth)
- Crimes (Amount of a Penalty Unit) Instrument 2026 (Cth)
- Privacy and Other Legislation Amendment Act 2024 (Cth)
- Commissioner Initiated Investigation into Clearview AI, Inc (Privacy) [2021] AICmr 54
- Clearview AI Inc and Australian Information Commissioner [2023] AATA 1069
- ‘AHM’ and JFA (Aust) Pty Ltd t/a Court Data Australia (Privacy) [2024] AICmr 29
- Court Data Australia and Office of the Australian Information Commissioner [2025] ARTA 876
- Office of the Australian Information Commissioner, Guidance on Privacy and Developing and Training Generative AI Models (Guidance, 21 October 2024)
- Office of the Australian Information Commissioner, Report into Preliminary Inquiries of I-MED (Report, 31 July 2025)
- Office of the Australian Information Commissioner, ‘Chapter B: Key Concepts’ in Australian Privacy Principles Guidelines (Guidelines, 21 December 2022)
- Office of the Australian Information Commissioner, ‘Chapter 3: APP 3 Collection of Solicited Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 13 May 2026)
- Office of the Australian Information Commissioner, ‘Chapter 6: APP 6 Use or Disclosure of Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 22 July 2019)
- Office of the Australian Information Commissioner, ‘Chapter 8: APP 8 Cross-Border Disclosure of Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 3 October 2025)
- Office of the Australian Information Commissioner, Guidance on Privacy and the Use of Commercially Available AI Products (Guidance, 21 October 2024)
- Office of the Australian Information Commissioner, ‘Statement on Clearview AI’ (Media Release, 21 August 2024)
- ‘Fines and Penalties’, Australian Securities and Investments Commission (Web Page)
- ‘Privacy Reform: Consultation on Exposure Draft Legislation’, Attorney-General’s Department (Web Page, 31 August 2026)
- Attorney-General’s Department, Privacy Reform (Consultation Paper, August 2026)