How to adapt knowledge testing and professional training to an AI world ?

Embracing AI in Professional Training and Testing

AI is sometimes perceived as a means of cheating during an exam, for example, by using ChatGPT, or as a way of impressing colleagues or supervisors with AI-generated code, as in the case of GitHub Copilot.

This situation serves as a reminder of the time when only basic calculators were allowed for French bachelor exams. The introduction of programmable calculators led to a delay of 20 years until 1996 when they were finally authorized. Even then, it wasn't until 2018 that regulations for specific calculators with restricted exam modes were put in place to differentiate between using a mini computer or a calculator only.

This demonstrates the slow pace of regulatory changes, which often lag behind technological advancements.

However, as AI continues to rapidly transform industries by automates certain tasks and becomes a fundamental part of productivity tools, it is crucial for individuals to develop new skills that complement AI, such as problem-solving, critical thinking, and creativity.

Professional training programs should focus on developing these skills to ensure that individuals are equipped to work alongside AI in the workplace, and not compete with AI.

Additionally, testing and certification should leverage AI to ameliorate the way we validate knowledge and skills

Some may question whether the use of ChatGPT should be allowed during a training or even a test, as it could be seen as a way to cheat or take shortcuts ?

A Real Life Example during Professional Training

I give some remote and on-premise trainings on Microsoft Azure using hosted labs exercises which required student to complete some tasks.

It’s common now that I notice that one of my students finishes the labs quicker than the others by using ChatGPT and argues that he has access to it at work and that my role is to make him efficient in real life environment.?

First, no need to ask him to swear that he will not use ChatGPT, a senior trainer will detect that quickly and dig the objectives of knowledge.?

I would evaluate if the student truly comprehends the lab's outcomes, identifies potential issues, identifies methods to optimize it, understands the advantages and disadvantages of the solution, and knows how to argue during a meeting.

This will test his problem-solving, critical thinking, and creativity skills.?

If the student displays a good understanding of these aspects, I would felicitate him for his work, as it reflects the well-known adage in the IT industry that "good programmers are sometimes the lazy ones." !

This saying highlights that a competent programmer will always search for the most efficient and effective way to achieve a task, which can include finding shortcuts or automating repetitive tasks.

It highlights the importance of working smarter with AI, not harder!

Why Open Book Certification Exams Are Outdated in the Age of AI

First, let's take a look at what can be eliminated with the possibility offered to students by AI: open book certification or renewal.

From my experience, I have to renew around 3 to 5 certifications each year, and I am not satisfied with some renewal processes that consist of responding to 20 questions while having access to the internet to be able to Google or ChatGPT the answers.

It is because it's 'easier to cheat' that this renewal has no or less value, and I am not quite confident that a simple renewal goes in the direction of continuous improvement of knowledge and the demonstration of it.

But does someone using ChatGPT during an open book examination is really cheating ?

For example, If we take a look at the Microsoft Certification Exam Agreement that you have to accept when renewing a certification using the Open book examination from Microsoft learning website, (not the certification test, but the simple renewal) it states that examples of misconduct include:

...

5. Submitting any work or answers that are not entirely Your own.

6. Providing or receiving improper assistance during the exam.

...

However, the use of ChatGPT during an open book examination raises whether it constitutes cheating.

Let's see the response of ChatGPT :

According to ChatGPT, it cannot be considered external assistance if the exam is open book and the use of external resources is permitted, but is his answer is not legal advice, I think that putting a specific statement for the use of AI is needed to clarify the situation.

I think that open book certification exams that allow test-takers to use any resource they want are outdated and do not provide a reliable measure of knowledge or capacity.

Today, to give more value to certifications, tests should be always taken in a controlled environment with AI-powered or human proctoring.

The Future of Certification Testing: AI-Powered Assessments

Here are some ways AI can modernize certification testing:

• Adaptive testing to validate all area of testing objectives
• Polymorphic Questions to render the memorization of brain dumps impossible and block paper certification holders.
• Natural Language Processing: AI can analyze written responses to open-ended questions for scenario-based assessments. This approach is closer to real-life situations where communication is done in writing, rather than simply checking a box
• Remote Proctoring with AI: a lightweight agent running in the web browser and monitoring new processes, keeping eyes on the screen using eye movement detection, no hiding the mouth, no talking. It will lower the cost of certification.
• Intelligent Insight: If you fail or pass, you will receive optional study recommendations to advance your skills.
• Knowledge Refresh Certification: to test more often and reflect the quick evolution of technologies
• Hands-on Labs Exercises : where test-takers are required to perform specific tasks or scenarios in a controlled environment, with the results checked by AI.

The Rise of Hands-On Certifications

Hands-on lab exercises should be integrated into all certification programs, mixed with knowledge and reasoning testing.

While some certifications have always been focused on real work, such as Cisco's, they were limited to testing specific knowledge or commands, and not adapted to validate reasoning.

I remember when Microsoft attempted to do this in its certification tests some years ago, but with no success, probably due to the complexity of validating the work done. However, this was before AI, and now the most popular and trending certifications are based on hands-on experience, designed to provide job proficiency on various roles.

With up-to-date training material, rigorous certification processes, and real-world exam lab environments, certified individuals will possess deep technical competency.

A good example of this is the proctored cybersecurity certification OSCP from Offensive Security, which has recently updated its rules to not allow the use of ChatGPT during the 24-hour lab exam.

While this may be a challenging experience, it provides greater value to the certification.

Conclusion

In conclusion, if the training focuses on problem-solving and critical thinking skills, then allowing the use of ChatGPT may not be desirable as it can provide quick answers without the necessary thinking and reasoning.?

However, if the goal is to train specific technical skills, then allowing the use of ChatGPT may be acceptable as it can help to quickly find the necessary information to complete the task.?

Certifications of knowledge should adapt to an AI world by using this technology to provide more value to the certification process or knowledge and skills.

Share your thoughts and experiences in the comments section below, and follow me on LinkedIn for more articles and industry news.

And do not forget to "Never Stop Learning" ... just like an AI system !

#artificialintelligence #learning #testing #certification #training