Hands on data analysis data reconstruction

Data refactoring

Introduce

After cleaning the data （ duplicate value 、 Missing value ）, Data can be reconstructed . Why should we do data reconstruction , Because in reality , It is possible that the data is distributed across multiple files ; Or some data in the data is relevant , Integration is required ; Need to get valid information from data （ Such as maximum value 、 Average and other statistical information ）, At this time, we need to use the related technology of data reconstruction .

Data merging

DataFrame The merger of

Include DataFrame Of merge and join、append

Used DataFrame Object merge Method , Here's the picture ：

meanwhile ,merge You can also merge by index

join： combination append It is more convenient to merge according to the index

Axial connection

There are two ways ：

• Use numpy：np.concatenat([arr, arr], axis=1)
• pd.concat()：

Grouping and aggregation of data

The data packet

Use groupby function

Data aggregation

• sum： Calculate the sum of the groups
• mean： Calculate the average of the groups
• max： Calculate the maximum value of the group

References

• merge in pandas
• concat in pandas
• groupby in pandas
• Hands on data analysis Chapter viii. and Chapter ten