Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ pyproject.toml
.devcontainer
Dockerfile
dev_guide.md
.pypirc
21 changes: 0 additions & 21 deletions LICENSE

This file was deleted.

38 changes: 26 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ with lightgbm. The R version of this package may be found
- Has efficient mean matching solutions.
- Can utilize GPU training
- **Flexible**
- Can impute pandas dataframes
- Can impute pandas dataframes and numpy arrays
- Handles categorical data automatically
- Fits into a sklearn pipeline
- User can customize every aspect of the imputation process
Expand All @@ -39,6 +39,7 @@ with lightgbm. The R version of this package may be found
- Data can be imputed in place to save memory
- Can build models on non-missing data


This document contains a thorough walkthrough of the package,
benchmarks, and an introduction to multiple imputation. More information
on MICE can be found in Stef van Buuren’s excellent online book, which
Expand Down Expand Up @@ -338,7 +339,7 @@ new_data_imputed = cust_kernel.impute_new_data(new_data=new_data)
print(f"New Data imputed in {(datetime.now() - start_t).total_seconds()} seconds")
```

New Data imputed in 0.040396 seconds
New Data imputed in 0.035129 seconds


## Saving and Loading Kernels
Expand Down Expand Up @@ -506,6 +507,19 @@ pd.DataFrame(optimal_params)


<div>
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}

.dataframe tbody tr th {
vertical-align: top;
}

.dataframe thead th {
text-align: right;
}
</style>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
Expand Down Expand Up @@ -561,10 +575,10 @@ pd.DataFrame(optimal_params)
</tr>
<tr>
<th>min_sum_hessian_in_leaf</th>
<td>0.1</td>
<td>0.1</td>
<td>0.1</td>
<td>0.1</td>
<td>0.01</td>
<td>0.01</td>
<td>0.01</td>
<td>0.01</td>
</tr>
<tr>
<th>min_gain_to_split</th>
Expand Down Expand Up @@ -811,7 +825,7 @@ kernel.plot_feature_importance(dataset=0)



![png](README_files/README_48_0.png)
![png](README_files/README_49_0.png)



Expand All @@ -824,7 +838,7 @@ kernel.plot_imputed_distributions()



![png](README_files/README_50_0.png)
![png](README_files/README_51_0.png)



Expand Down Expand Up @@ -871,7 +885,7 @@ acclist
0 0.35
1 0.81
2 0.81
3 0.78
3 0.84
Name: Species Imputation Accuracy, dtype: float64


Expand Down Expand Up @@ -1021,7 +1035,7 @@ plot_matrix(dat, dat.columns)



![png](README_files/README_60_0.png)
![png](README_files/README_61_0.png)



Expand Down Expand Up @@ -1054,7 +1068,7 @@ kernel_mean_match.plot_imputed_distributions()



![png](README_files/README_63_0.png)
![png](README_files/README_64_0.png)



Expand All @@ -1065,7 +1079,7 @@ kernel_no_mean_match.plot_imputed_distributions()



![png](README_files/README_64_0.png)
![png](README_files/README_65_0.png)



Expand Down
Binary file added README_files/README_49_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added README_files/README_51_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added README_files/README_61_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified README_files/README_64_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added README_files/README_65_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 14 additions & 14 deletions README_gen.ipynb

Large diffs are not rendered by default.

12 changes: 5 additions & 7 deletions miceforest/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,11 @@ def ampute_data(
amputed_data = data.copy()
num_rows = amputed_data.shape[0]
amp_rows = int(perc * num_rows)
random_state = ensure_rng(random_state)
rs = ensure_rng(random_state)
variables = list(data.columns) if variables is None else variables

for col in variables:
ind = random_state.choice(amputed_data.index, size=amp_rows, replace=False)
ind = rs.choice(amputed_data.index, size=amp_rows, replace=False)
amputed_data.loc[ind, col] = np.nan

return amputed_data
Expand Down Expand Up @@ -91,7 +91,7 @@ def stratified_subset(

"""

random_state = ensure_rng(random_state=random_state)
rs = ensure_rng(random_state=random_state)

cat = False
if y.dtype.name == "category":
Expand All @@ -112,9 +112,7 @@ def stratified_subset(
digits_s = (digits_p * size).round(0).astype("int32")
diff = size - digits_s.sum()
if diff != 0:
digits_fix = random_state.choice(
digits_i, size=abs(diff), p=digits_p, replace=False
)
digits_fix = rs.choice(digits_i, size=abs(diff), p=digits_p, replace=False)
if diff < 0:
for d in digits_fix:
digits_s[d] -= 1
Expand All @@ -128,7 +126,7 @@ def stratified_subset(
d_v = digits_v[d_i]
n = digits_s[d_i]
ind = np.where(digits == d_v)[0]
choice = random_state.choice(ind, size=n, replace=False)
choice = rs.choice(ind, size=n, replace=False)
sub[added : (added + n)] = choice
added += n

Expand Down
Loading