Purposely not disclosing the usage of a tool that is widely reported and known to provide false information until after it is signed into law shows a pretty clear lack of ethics.

The problem is GTP is learning from our existing knowledge base. If legislation is trying to amend a broken system, we don't want AI to be modeling that system. This case seems fairly harmless, an AI takeover isn't what we should be worries about.

Something like institutional racism being replicated in a more insidious manner is the concern. Relying on these closed systems potentially gives the type of people who implemented the discrimination being modeled to turn around and say, 'See, we were right all along!' If results are held up on a pedestal and AI is integrated into our political and legal systems, it may make changing society for the better much harder.

We shouldn't universally condemn tools like ChatGTP being used in this way, but we should tread very carefully when it comes to large scale societal changes.

As long as he fact checked what chatgpt told him, who cares?

But he can't fact check it - he was using it because he didn't know enough about the topic to write that section in the first place.

I was suggesting ChatGPT to a social worker I know who was complaining about the number of reports he has to write - AI could bang out a first draft that he could then work up into the final report because he knows the cases and the relevant laws and procedures. If he didn't know the subject inside and out it could be dangerous. Even then, he should flag it up for others to look it over in case of hallucinations.

All joking aside, this is the answer.