Data Analytics with python/[Data Analysis]
[Text]S8_04_Text_cleaning(removing_punctuation)
보끔밥0130
2023. 1. 21. 21:55
728x90
In [39]:
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
Out[39]:
'!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~'In [27]:
echo_df = pd.read_csv('Echodot2_Reviews.csv', encoding='utf-8')
echo_df.head()
Out[27]:
| Rating | Review Date | Configuration Text | Review Text | Review Color | Title | User Verified | Review Useful Count | Declaration Text | Pageurl | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 3 | 10/3/2017 | Echo Dot | Not great speakers | Black | Three Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 1 | 4 | 9/26/2017 | Echo Dot | Great little gagit | White | Four Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 2 | 5 | 9/8/2017 | Echo Dot | Awesome 👏🏽 | White | Awesome! | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 3 | 5 | 10/19/2017 | Echo Dot | Love my Echo | Black | Five Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 4 | 5 | 9/17/2017 | Echo Dot | Great device | Black | Five Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
In [30]:
echo_df.dropna(subset=['Review Text'], how='any', axis = 0, inplace=True)
In [36]:
# contains the word 'love'
mask = echo_df['Review Text'].str.lower().str.contains('love')
echo_df[mask]
Out[36]:
| Rating | Review Date | Configuration Text | Review Text | Review Color | Title | User Verified | Review Useful Count | Declaration Text | Pageurl | |
|---|---|---|---|---|---|---|---|---|---|---|
| 3 | 5 | 10/19/2017 | Echo Dot | Love my Echo | Dark Black | Five Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 11 | 5 | 10/9/2017 | Echo Dot | Alexa...You rock!! OMG people. I am not a tech... | Dark Black | This is the greatest thing since chocolate | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 13 | 5 | 9/20/2017 | Echo Dot | I love using Alexa with the smart outlets for ... | White | Love it! | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 17 | 5 | 10/8/2017 | Echo Dot | Cant say enough !!I love my DOT!!!! | Dark Black | Happy with her Dot | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 20 | 5 | 9/8/2017 | Echo Dot | Love the echo! | Dark Black | This is the second one for the house. | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 6845 | 5 | 9/13/2017 | Echo Dot | We now have 4 Dots & one Show in the house. Pe... | White | Perfect for everyone in our family | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 6846 | 5 | 9/4/2017 | Echo Dot | Alexa is exceptional, I am getting use to ever... | Dark Black | From what I know so far I love it. | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 6849 | 5 | 9/14/2017 | Echo Dot | Love it. | Dark Black | Five Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 6850 | 5 | 9/17/2017 | Echo Dot | This is so much fun! I love her. | Dark Black | In love with Alexa!! | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
| 6853 | 5 | 9/27/2017 | Echo Dot | I have now set Alexa up to control lights in m... | Dark Black | Simply fabulous! | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... |
1963 rows × 10 columns
In [40]:
# String module contains constants and classes for working with text
import string
string.punctuation
Out[40]:
'!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~'In [42]:
Test = '$I love Pandas & Data Anlaytics!!'
Test_removed_punc = [ char for char in Test if char not in string.punctuation ]
Test_punc_join = ''.join(Test_removed_punc)
Test_punc_join
Out[42]:
'I love Pandas Data Anlaytics'In [43]:
# A function to remove punctuations
def remove_punc(text):
Test_removed_punc = [ char for char in text if char not in string.punctuation ]
Test_punc_join = ''.join(Test_removed_punc)
return Test_punc_join
In [44]:
remove_punc(Test)
Out[44]:
'I love Pandas Data Anlaytics'In [45]:
echo_df['reviews_unpunc']= echo_df['Review Text'].apply(remove_punc)
echo_df
Out[45]:
| Rating | Review Date | Configuration Text | Review Text | Review Color | Title | User Verified | Review Useful Count | Declaration Text | Pageurl | reviews_unpunc | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 3 | 10/3/2017 | Echo Dot | Not great speakers | Dark Black | Three Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | Not great speakers |
| 1 | 4 | 9/26/2017 | Echo Dot | Great little gagit | White | Four Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | Great little gagit |
| 2 | 5 | 9/8/2017 | Echo Dot | Awesome 👏🏽 | White | Awesome! | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | Awesome 👏🏽 |
| 3 | 5 | 10/19/2017 | Echo Dot | Love my Echo | Dark Black | Five Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | Love my Echo |
| 4 | 5 | 9/17/2017 | Echo Dot | Great device | Dark Black | Five Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | Great device |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 6850 | 5 | 9/17/2017 | Echo Dot | This is so much fun! I love her. | Dark Black | In love with Alexa!! | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | This is so much fun I love her |
| 6851 | 5 | 9/16/2017 | Echo Dot | I'm having a lot of fun with it. | White | Five Stars | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | Im having a lot of fun with it |
| 6852 | 3 | 9/26/2017 | Echo Dot | I bought this as a gift for my husband and he ... | Dark Black | We would still like to exchange for something ... | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | I bought this as a gift for my husband and he ... |
| 6853 | 5 | 9/27/2017 | Echo Dot | I have now set Alexa up to control lights in m... | Dark Black | Simply fabulous! | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | I have now set Alexa up to control lights in m... |
| 6854 | 2 | 9/12/2017 | Echo Dot | What a shame, I tried one my friend has and wa... | Dark Black | Bummer, didnt even make it through its voice t... | Verified Purchase | NaN | NaN | https://www.amazon.com/All-New-Amazon-Echo-Dot... | What a shame I tried one my friend has and was... |
6852 rows × 11 columns
In [52]:
# check if the function worked as expected
echo_df['Review Text'][25]
Out[52]:
"I bought this as an extension of the Echo (I use the main unit in my living room, and the Dot in my bedroom). It's very accurate in it's response, the sound is great for it's size, and it makes things that I would normally take minutes or longer almost instantaneous. Beware, because it makes so much in your life so easy, there may be a risk of getting lazy and entitled."In [53]:
echo_df['reviews_unpunc'][25]
Out[53]:
'I bought this as an extension of the Echo I use the main unit in my living room and the Dot in my bedroom Its very accurate in its response the sound is great for its size and it makes things that I would normally take minutes or longer almost instantaneous Beware because it makes so much in your life so easy there may be a risk of getting lazy and entitled'728x90