Issue
I would like some help/direction to find out how many times a set of ranges appears in an array of min/max values.
import numpy as np; import pandas as pd
np.random.seed(42)
df = pd.DataFrame(np.random.rand(1000, 2),
columns=["Min", "Max"]) \
.mul(6).query("Min <= Max").reset_index(drop=True)
lb = np.arange(1, 5, 0.1)
ub = np.arange(1, 5, 0.1)
a = df[['Min','Max']].to_numpy()
factor = 1
arry = []
My first attempt would be to convert the min/max into a numpy array and iterate over all the arrays to produce the output.
factor = 1
arry = []
for row in a:
for x in lb:
for y in ub:
lbExists = 0
rangeExists = 0
if y <= x:
continue
if x >= row[0] and x <= row[1]:
lbExists = 1
if x >= row[0] and y <= row[1]:
rangeExists = 1
project = 0
if lbExists == 1 and rangeExists == 0:
project -= factor
if lbExists == 1 and rangeExists == 1:
project = (y - x) * factor
arry.append([round(x, 2), round(y, 2), row[0], row[1], lbExists, rangeExists , round(project,2)])
Output (truncated):
[[1.0, 1.1, 1.34, 5.8, 0, 0, 0],
[1.0, 1.2, 1.34, 5.8, 0, 0, 0],
[1.0, 1.3, 1.34, 5.8, 0, 0, 0],
[1.0, 1.4, 1.34, 5.8, 0, 0, 0],
[1.0, 1.5, 1.34, 5.8, 0, 0, 0],
[1.0, 1.6, 1.34, 5.8, 0, 0, 0],
[1.0, 1.7, 1.34, 5.8, 0, 0, 0],
[1.0, 1.8, 1.34, 5.8, 0, 0, 0],
[1.0, 1.9, 1.34, 5.8, 0, 0, 0],
[1.0, 2.0, 1.34, 5.8, 0, 0, 0],
...
Then, I would like to do some analysis to find how many times each range appears in the set.
Is there a better way to do this?
Solution
A much faster method of computation is using various numpy
methods.
# array of all valid permutations of x and y
arr_xy = np.array(np.meshgrid(lb.round(2), ub.round(2))).T.reshape(-1, 2)
arr_xy = arr_xy[arr_xy[:, 0] < arr_xy[:, 1]]
# lbExists - (x >= row[0] and x <= row[1])
lbExists = (arr_xy[:, 0][:, np.newaxis] >= a[:, 0]) * \
(arr_xy[:, 0][:, np.newaxis] <= a[:, 1])
# rangeExists - (x >= row[0] and y <= row[1])
rangeExists = lbExists * (arr_xy[:, 1][:, np.newaxis] <= a[:, 1])
# project based off of lbExists initially (else 0)...
project = np.where(lbExists,
# then on value of rangeExists
np.where(rangeExists,
# (y - x) * factor if true
np.tile(np.diff(arr_xy) * factor, a.shape[0]),
# else -1 * factor
np.full(rangeExists.shape, -factor)),
0)
# combine variables
arr_out = np.hstack([
# permutations of upper and lower bound
np.vstack([arr_xy] * a.shape[0]),
# repeated values of Min and Max
np.repeat(a, arr_xy.shape[0], axis=0),
# lbExists 2d -> 1d
lbExists.T.reshape(-1)[:, np.newaxis],
# rangeExists 2d -> 1d
rangeExists.T.reshape(-1)[:, np.newaxis],
# project 2d -> 1d
project.T.reshape(-1)[:, np.newaxis].round(2)])
I have added comments throughout the code, but do ask if there is anything that needs clarifying.
The difference in timings (about x100 faster):
# your loop in the question
5.84 s ± 295 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
# this method
50.2 ms ± 3.49 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
And to confirm that these are the same:
(arr_out==np.array(arry)).all()
#Out: True
Answered By - Rawson
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.