Prepare your data for ML | Text Classification Tutorial Pt. 1 (Coding TensorFlow)

No video

Prepare your data for ML | Text Classification Tutorial Pt. 1 (Coding TensorFlow)

Рет қаралды 89,434

Күн бұрын

@lmoroney is back with another episode of Coding TensorFlow! In this episode, we discuss Text Classification, which assigns categories to text documents. This is part 1 of a 2 part sub series that focuses on the data and gets it ready to train a neural network. Laurence also explains the unique challenges associated with Text Classification. Watch to follow along and stay tuned for part 2 of this episode where we’ll look at how to design a neural network to accept the data we prepared.
Hands on tutorial → bit.ly/2CNVMbi
Watch Part 2 • Designing a neural net...
Subscribe to TensorFlow → bit.ly/TensorFlow1
Watch more Coding TensorFlow → bit.ly/2zoZfvt

Пікірлер: 50

@daneshj4013 3 жыл бұрын

Basically the most important part was already done for you...can you make a video on how to format the text to number repretations because there won't always be a third party that would have conveniently done this for us...

@user-eb5sm8ec8r 3 жыл бұрын

check the zero to hero course on tensorflow, then. the tokenization and sequencing process is covered there.

@MsVsmaster 2 жыл бұрын

Just split the sentences into words and for every word you append to a pair or map, where every pair is [word, unique index], then you have a dictionary where you can scale down up whenever you want without a lib, super simple, also you can make the same approach to save reserved index for pads, start etc.

@TonyStark-lv8kv 3 жыл бұрын

Conversion of words to numbers has already been done here. That's the most important part

@user-eb5sm8ec8r 3 жыл бұрын

check the zero to hero course on tensorflow, then. the tokenization and sequencing process is covered there.

@Otonium 5 жыл бұрын

Nice video, please include links to the following videos, sometimes is confusing even with playlists

@afodiaayuba9917 5 жыл бұрын

how to convert text to vectors

@laurencemoroney655 4 жыл бұрын

A process called 'Embeddings' or 'Word2Vec'

@robind999 5 жыл бұрын

Yes, this is what I was looking for. thank you.

@Hamza-ny7rx 6 жыл бұрын

o o o that's what i was waiting.

@laurencemoroney655 5 жыл бұрын

Great! Hope it was helpful :)

@amfakh. 5 жыл бұрын

and no one talks about how to convert actual text to integer

@mattimatilainen8437 5 жыл бұрын

Did you watch the video?

@tibfox 4 жыл бұрын

@@mattimatilainen8437 I did and no one talks about it

@sharif47 4 жыл бұрын

NLP Zero to Hero series (also by this guy) covers it. Here's a brief so you know whether it's what you're looking for: 1. Fit a Tokenizer on training Text so that it creates a dictionary with unique integer value for every unique word in the training data. (Words with earlier appearance probably get lower number) 2. Use text_to_sequences on your (train/test) data with the help of that tokenizer to convert your sentence into a list of integers. If this doesn't answer your question, then I'm afraid I cannot help you anymore.

@kanhataak1269 3 жыл бұрын

Hello, text Classification problem one text feature witch have text and task is make a category feature like classify the users into suitable categories. how to solve

@GregFrenchChannel 6 жыл бұрын

Thanks Laurence! I love these videos!

@laurencemoroney655 5 жыл бұрын

Thanks!

@AbhishekKumar-mq1tt 6 жыл бұрын

Thank u for this awesome video and series, I am waiting for next video

@laurencemoroney655 5 жыл бұрын

Thanks! :)

@niroopreddy6910 4 жыл бұрын

can anyone point me in the direction of converting my own csv that has couple of text columns(features) and convert them to a usable dataset. The csv file is pretty large and it has millions of rows, in which i have categorical data as well as the text data.

@laurencemoroney655 4 жыл бұрын

Take a look at TFDS (TensorFlow Datasets)

@farhadshadmand5785 5 жыл бұрын

I have myself data, I want to cover them to code, you always from movie code that it is ready before. I want to know how happened there.

@user-eb5sm8ec8r 3 жыл бұрын

check the zero to hero course on tensorflow, then. the tokenization and sequencing process is covered there.

@naz-kh6lj 4 жыл бұрын

hi , i would like to ask you something. what techniques should i use to find some keyword in my csv file and then if match with the keyword, i want to assign it to another keyword. the output something like this, column A Keyword DUMPBLT:TESTING FAILED: OPERATOR PUSHED STOP BUTTON SYSTEM FAILED if i found keyword of 'DUMPBLT' and 'PUSHED STOP BUTTON' in column A, i want to assign it to "SYSTEM FAILED" and put to other column. can you help me about this ?

@davidsteinbruch 5 жыл бұрын

How can you determine which words are going to be part of the index?

@AnujaTayal 3 жыл бұрын

There is a major bug- When we are shifting word_index, technically we should shift train data integers too.

@fatinebenachour9045 4 жыл бұрын

Hello, can someone please tell me how to get the most and least common word that he was talking about. Thank you

@rajanandj1177 2 жыл бұрын

hands on tutorial page is not found

@satadruhazra8607 4 жыл бұрын

I'm trying to build an intent classification model with tensorflow.. I'm facing some issues about validation accuracy and prediction accuracy.. I want some expert advice. can you provide your linkedin link or any contact info to help me out Please.

@Trackman2007 5 жыл бұрын

Is it new version of Jupyter notebook, other tool or some plugins?

@ursamajor472 Жыл бұрын

Hands on tutorial link is not working :(

@maglionejm 2 жыл бұрын

Hi! Link to code is broken, could you provide a new one?

@zoecarlibur 5 жыл бұрын

I mostly do frontend stuff, But I like how accessible TensorFlow is. I wonder if it's possible to do Machine learning and Front End Engineering at the same time.

@ardianwhy 4 жыл бұрын

TensorFlow.js

@TorpisoulYT 4 жыл бұрын

I need this for JavaScript

@JaydipBiniwale 6 жыл бұрын

Hey Laurence, I am facing an issue with how to use TF-Hub for the embedding part of the classification and then serving the model as a canned estimator. The variable weights of the frozen model are greater than 250 MB, and thus cannot be used on the cloud Machine learning engine to serve my model.

@wisdal 5 жыл бұрын

Part 2 please!

@laurencemoroney655 5 жыл бұрын

Part 2 now available :)

@laurencemoroney655 5 жыл бұрын

Where does the 250Mb limit come from? Sorry, a little confused.

@JaydipBiniwale 5 жыл бұрын

In the link below, the documentation says you can't upload a model bigger than 250 MB of size and due to the tf-hub module the size of the canned estimator goes beyond that. cloud.google.com/ml-engine/docs/tensorflow/deploying-models

@denissetiawan3645 5 жыл бұрын

is it necessary to pad and trim the data? what's the use for it?

@laurencemoroney655 5 жыл бұрын

So that all reviews are the same length.

@denissetiawan3645 5 жыл бұрын

i mean the fundamental reason for it has to be the same length

@stacksonchain9320 5 жыл бұрын

TensorFlow needs the input data as a multidimensional array(Tensor) or in simpler terms a matrix mxn with m being the number of reviews and n being the words. (note that m and n are fixed and cannot vary). you can either one hot encode the data similar to sklearn or just pad(put a limit on the maxlength). padding is more complicated code-wise, but less computationally expensive.

@SGTPOOKEY 5 жыл бұрын

Does anyone know how they (keras) converted the IMDB dataset into integers?

@itztobii 5 жыл бұрын

Hi there! Have you found a solution for the integers? I'm working on a project for speach analysis via classification. Maybe you can help me? :) pls PN me!

@ahaduzzamanmunna9035 4 жыл бұрын

Where is the tutorial for using Google Console to upload, train and test data? Does google care to show step by step instruction on how to prepare data? It's silly that for text classification people need to split data by label and put in separate folders. Please provide some basic tutorials first on how to use Google console and then give tutorial on writing code, etc.