A bit old now, here is some updated code for anyone watching. import random import json import pickle import numpy as np import tensorflow as tf import nltk from nltk.stem import WordNetLemmatizer lemmatizer = WordNetLemmatizer() intents = json.loads(open('intents.json').read()) words = [] classes = [] documents = [] ignoreLetters = ['?', '!', '.', ','] for intent in intents['intents']: for pattern in intent['patterns']: wordList = nltk.word_tokenize(pattern) words.extend(wordList) documents.append((wordList, intent['tag'])) if intent['tag'] not in classes: classes.append(intent['tag']) words = [lemmatizer.lemmatize(word) for word in words if word not in ignoreLetters] words = sorted(set(words)) classes = sorted(set(classes)) pickle.dump(words, open('words.pkl', 'wb')) pickle.dump(classes, open('classes.pkl', 'wb')) training = [] outputEmpty = [0] * len(classes) for document in documents: bag = [] wordPatterns = document[0] wordPatterns = [lemmatizer.lemmatize(word.lower()) for word in wordPatterns] for word in words: bag.append(1) if word in wordPatterns else bag.append(0) outputRow = list(outputEmpty) outputRow[classes.index(document[1])] = 1 training.append(bag + outputRow) random.shuffle(training) training = np.array(training) trainX = training[:, :len(words)] trainY = training[:, len(words):] model = tf.keras.Sequential() model.add(tf.keras.layers.Dense(128, input_shape=(len(trainX[0]),), activation = 'relu')) model.add(tf.keras.layers.Dropout(0.5)) model.add(tf.keras.layers.Dense(64, activation = 'relu')) model.add(tf.keras.layers.Dropout(0.5)) model.add(tf.keras.layers.Dense(len(trainY[0]), activation='softmax')) sgd = tf.keras.optimizers.SGD(learning_rate=0.01, momentum=0.9, nesterov=True) model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy']) model.fit(trainX, trainY, epochs=200, batch_size=5, verbose=1) model.save('chatbot_model.h5') print('Done')

Instructions unclear, made Ultron. 😂

Man, you are THE ONLY ONE explaining this intent type of neural net-based chatbot. Thanks a lot

Awesome content! when are we getting the biceps tutorial tho?

I love your videos! They're so useful :D I've just purchased your 7 in 1 book from amazon

Videos on neural network theory would be fantastic! Thanks again for another great video!

sooo interested to the neural network theory!!! Love this channel so much, i've learned so much, thank you!!

I would love to see a video about the theory, great video, thank you:)

Thank you. Great commentary and clear explanations. Easy to follow how you built up the application. Perfect.

A much-needed updated vision of the chatbot from Tech with Tim. Love this video. From a viewer standpoint and someone who is coding along with you to program this thing. Either don't put a video of yourself on the screen, put it in the upper right-hand corner, or make it smaller. It's hard to code along with you as the camera of yourself blocks some of your code in the video.

This was amazing! Thank you so much I have learned a lot :D

This is an amazing tutorial brother, thanks for all the knowledge you've shared! You earned a subscriber!

? - keras in "tensorflow.keras..." is not recognized by 'Pycharm', but it works anyway. Changes: - from tensorflow.keras.optimizers import SGD + from tensorflow.keras.optimizers.legacy import SGD - training = np.array(training) +training = np.array(training, dtype=object) -sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True) +sgd = SGD(learning_rate=0.01, decay=1e-6, momentum=0.9, nesterov=True)

Man, you are so underrated. I always search for simple examples of complex topics. After looking at your videos I am able to implement the basics in my projects easily

Hey bro, your programming style is sophistecated 🧑‍💻

Ah! Was waiting for this only!🔥

Never seen this type of content before, really amazing

Thanks 👍 I was actually waiting for this.

Well done. As I am new to chatbot() and python. Your video helped to fill in all the gaps in Tim video. Thanks brother.

Bro You are one of the most Underrated YT channel I've ever seen..... Keep up the good work : - )

when it comes to trading in the various financial markets the relevance of professional guidance should not be undervalued as it could be the key to any traders success in profit making

#Dec 2023- import random import json import numpy as np import pickle import tensorflow as tf import nltk from nltk.stem import WordNetLemmatizer from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, Activation, Dropout from tensorflow.keras.optimizers import SGD from tensorflow.keras.preprocessing.sequence import pad_sequences # Download NLTK data nltk.download('punkt') nltk.download('wordnet') lemmatizer = WordNetLemmatizer() intents = json.loads(open('intents.json').read()) words = [] classes = [] documents = [] ignore_letters = ['?', ',', '!', '.'] for intent in intents['intents']: for pattern in intent['patterns']: word_list = nltk.word_tokenize(pattern) documents.append((word_list, intent['tag'])) words.extend(word_list) # Append words to the 'words' list if intent['tag'] not in classes: classes.append(intent['tag']) words = [lemmatizer.lemmatize(word.lower()) for word in words if word not in ignore_letters] words = sorted(set(words)) classes = sorted(set(classes)) pickle.dump(words, open('words.pkl', 'wb')) pickle.dump(classes, open('classes.pkl', 'wb')) training = [] output_empty = [0] * len(classes) for document in documents: bag = [] word_patterns = document[0] word_patterns = [lemmatizer.lemmatize(word.lower()) for word in word_patterns] # Create a bag of words with 1s and 0s bag = [1 if word in word_patterns else 0 for word in words] # Create the output row output_row = list(output_empty) output_row[classes.index(document[1])] = 1 # Append the bag and output_row as separate lists training.append([bag, output_row]) # Shuffle the training data random.shuffle(training) # Split the training data into X and Y train_x = np.array([item[0] for item in training]) train_y = np.array([item[1] for item in training]) model = Sequential() model.add(Dense(128, input_shape=(len(train_x[0]),), activation='relu')) model.add(Dropout(0.5)) model.add(Dense(64, activation='relu')) model.add(Dropout(0.5)) model.add(Dense(len(train_y[0]), activation='softmax')) sgd = SGD(learning_rate=0.01, momentum=0.9, nesterov=True) model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy']) hist = model.fit(train_x, train_y, epochs=200, batch_size=5, verbose=1) model.save('Chatbot_model.h5', hist) print("Done!")

Great video, I learned so much, thank you!

brother please do not stop! you are awesome

I have to make one for school so this is perfect timing ;^)

I never opened a KZfaq notification so fast

Hey manx nice content, always excited when you post a new video

Awesome tutorial, it worked right away! You definitely got me as a subscriber.

Can you make a series on neural network theory? This video was very helpful.Thanks

20:42 very interested in the theory, I like how you explain and would be cool to hear it from you

This tutorial is fantastic! Thank you for sharing.

Amazing tutorial, thanks for the lesson.

Man do i love that intro 🔥

Hi NeuralNine, this is a great piece. I would like to ask you if this tutorial would work for another language. Would it be enough to just edit intents, to comply with the language I would like to use? And the second question, do you have some tutorial, on how to implement this bot to streamlabs chatbot for twitch?

Incredible tutorial, thanks!

This helped me learn a lot about neural networks thanks!

Your videos are always amazing, always i learn a lot from your videos 🤩🤩

So...as far as the machine learning goes, it's just a text classification algorithm. I would love to see a version of this with actual text generation as well!

Thanks for the Video Florian! 👏👏

As usual great video, excelent channel

Awesome work bro! I like your videos Kudos to you I suggest you make a tutorial video on show to deploy an AI CHATBOT to back-end server to improve responsiveness and present more details.

this tutorial is very useful! can I have the project source code for self-learning?

UNDERRATED , u deserve a million sub

man I was just building a hard-coded chatbot, this vid came in the right time for me. thanks bro

The video got really good but we don’t really clarify what packages to download. I know you said what at the beginning at Imports but it still gives me error messages with Tensorflow. For example: ' To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. '

NIce video! Can you make an in-depth tutorial on making a friend-like chatbot that uses neural networks and NLP, and adapts it to modern libraries like Keras and TensorFlow? I've been making a project, but I've been having some issues.

This video is amazing ! thank you!

i have'nt watched your video before this video but now ican't stop watching your video

it would be cool if you made a video where you combined a chatbot like this with/into a discord bot!

When will you open a Discord Server?

Thank you for such a wonderful content !

The code is not working. output_row[classes.index(document[1])] = 1 IndexError: list index out of range

Hey NeuralNine, really like the videos. Could you please like zoom in to the code... Or like increase the font. Its kinda difficult to see the code especially when not on full screen. Even when youre showing other stuff like how to install mingw for c++... The text looks really small But i really love your content and learn a lot. So if you could address that i'd be stoked

This video is really great and this AI projects series is extraordinary

Please put more videos like this bro, I needed some sorry more.... And if you are about to launch a video give a gif for that. That thrill will be enough for me to be happy. Don't get faded bro, we need you...

Hey, any idea how you'd get this to run with tflite? Edit: Okay, I've gotten the model to run, but I realised nltk stemming is really slow. Is there a faster alternative? (actually I'll try figure out exactly what's taking ages) Edit #2: Nevermind, it all works, there's just a delay the first time the process is called for some reason (Which I'll try to trace and fix) Thanks for the great tutorial, can't wait to setup my own personal assistant!

for intent in intents['intents']: for pattern in intent['patterns']: word_list = nltk.word_tokenize(pattern) words.append(word_list) documents.append((word_list, intent['tag'])) if intent['tag'] not in classes: classes.append(intent['tag']) This showing me error " For pattern in intent['patterns'] KeyError : 'patterns' Please help me out!

Thanks, this video helped me a lot!

Okay, this is the best tutorial on chat bots!! i can follow along! Quick question: Is it possible to show the "questions" as options to the user so that instead of typing, they choose from the options of questions instead? i tried following Tech with Tim's but it was kinda hard.

Hi, great content! Although, is it possible for you to upload the code in your videos to a github repo? Thanks

Wao that's made my day.... Thanks a lot for sharing such an insightful video

really a great tutorial, nice job

When I added training data to the intents.json file, I got this error, anybody who can help? ValueError: Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 38 but received input with shape (None, 84) I figured out that the 84 is just the amount of patterns in the intents.json file, but I haven't figured out why the input shape has to have a value of 38.

and this is how my creator came to life.

Thank you very much sir, for this awesome tutorial....😍😍😍😍

Very excellent work and Good amount of stuff there in for a beginners to have. I am learning Paython now.

Ah yes, now I can make a very own best friend who can understand me :DDDD

Can you please provide the files of the bot (because mine didn't worked)

Thank You All I needed this.

a series about NN/ ai in general would be dope

there is correction on line 51 in chatbot.py ."responses" should be replaced with "response"

Works great! Im having this error when adding more than 3 intents though. ValueError: Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 27 but received input with shape [None, 30]. :/

awesome tutorial bro!

WOW very good, excelent explanation

Hello... i have tried to copy your code now in 2023 and apparently there is an issue with training = np.array(training) since the [bag, output_row] columns don't have the same dimensions (maybe I have copied something wrong along the way...). I have came up with a solution that bypass the issue, although its quite a caveman style solution and I do not have the knowledge to understand its consequences, but it does not crash the code and allows me to train the chatbot. solution: when defining the dimensions of: output_empty = [0] * len(classes) instead do: output_empty = [0] * len(words) this way the dimensions of "bag" and "output_row" are the same. (It will fill the space with 0, so it shouldn't change the bot training accuracy... but what do I know :D )

training = np.array(training) is returning an error, can you help? --------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[24], line 26 24 # shuffle our features and turn into np.array 25 random.shuffle(training) ---> 26 training = np.array(training) 28 # create train and test lists. X - patterns, Y - intents 29 train_x = list(training[:, 0]) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (27, 2) + inhomogeneous part.

For anyone having issues with the model not running. change train_x and train_y into a tensor and adjust the input shape of the model to be 15, 5

What is the difference between this and using something like the difflib SequenceMatcher library or the Levenshtein method, as in finding the similarity metric between the given input and a list of sentences and then picking the closest match as a response by looping through all the data one by one? Just tried it, and it works exactly the same but with a few lines of code, which brings me to the question why using AI for this is even necessary?

Well, I followed along and as I ran it....this guy decided to reply "Hello" with "Talk to you later" LMAOO 😭Even a bot isnt interested in talking with me 😒

my bot keeps saying the intent is goodbye even if I say "how are you?" it says hey or goodbye or talk to you later

I learned a lot through this video. However since we have static replies for this exact project I believe that splitting the string from the user input into a list as you did, finding a key word and thus give the reply could be easily and faster done with if else statements. I get the educational manner for the AI part but because you didn’t exactly analyze the code that you used in the machine learning part because of the purpose of the video I will stick to my opinion having the thought of the purpose of the video being “build a chatbot” and not “use ai to create something”. Great video though❤

Nice tutorial on basic Keras NN setup. The truth is there is no need for a NN here. What it does is a simple input text selector and a random generator to output the response text. You could do the same using regexp/similar text/soundex word matching.

Hello, i am unable to install tensorflow!!please help

can i have the source code?

Yes I’m very interested in neural network theory! 🙋‍♂️🙋‍♂️🙋‍♂️🙋‍♂️🙋‍♂️

Thanks for this tutorial video.

thanks, that works(some code mistakes corrects by chatGPT) nice video

You helped me lot bro😮

22:08 Hi, i'm new to DL.. When we use Softmax, if there are (say)two outputs in the network, it will output probabilities like [.4 ,.6] or [.1, .9] or [.5, .5] etc right? (sum always 1).. so even if we enter some gibberish as input, there will always be some output(s) above error threshold of (say) .25. How ill I tweak this so this won't happen? ( i mean if i enter "ahfliehf fsdlihfasdh" to bot, the outputs should be like something like [0.001, 0.00123])

Great tutorial!

I'm trying to create a chatbot for customer care, it is going to be a live support agent for industries like Telecom companies, so after I finish with all the knowledge you have shared maybe I will have some questions and everything else.

if you get an error with hist = model.fit(np.array(train_x), np.array(train_y), epochs=200, batch_size =5, verbose=1 ), just delete "accuracy in model.compile

espero que puedas traducir este mensaje, pero solo quiero agradecerte porque me ayudaste mucho al basarme en este bot, saludos desde Mexico :)

Thanks a lot Sir for the simplified explanation. Kudos to you for your great work !!!!!!!!!!!

bro your intro is fire

Vielen Dank für dein Video, sehr interessant und nützliches Informationen

Awsome vidéo, usefull !!

thank u very much keep it up💯

Are you ever going to make a tutorial on a chatbot that generates its own responses?

when i finished this chat i typed in terminal: " hello!" the answer was "sad to see you go :( " hahahahahahah i'm crying !!!! thank you it's working perfectly