Having issue filtering my result dataframe with an or
condition. I want my result df
to extract all column var
values that are above 0.25 and below -0.25.
This logic below gives me an ambiguous truth value however it work when I split this filtering in two separate operations. What is happening here? not sure where to use the suggested a.empty(), a.bool(), a.item(),a.any() or a.all()
.
result = result[(result['var'] > 0.25) or (result['var'] < -0.25)]
The
or
andand
python statements requiretruth
-values. Forpandas
these are considered ambiguous so you should use “bitwise”|
(or) or&
(and) operations:These are overloaded for these kind of datastructures to yield the element-wise
or
(orand
).Just to add some more explanation to this statement:
The exception is thrown when you want to get the
bool
of apandas.Series
:What you hit was a place where the operator implicitly converted the operands to
bool
(you usedor
but it also happens forand
,if
andwhile
):Besides these 4 statements there are several python functions that hide some
bool
calls (likeany
,all
,filter
, …) these are normally not problematic withpandas.Series
but for completeness I wanted to mention these.In your case the exception isn’t really helpful, because it doesn’t mention the right alternatives. For
and
andor
you can use (if you want element-wise comparisons):numpy.logical_or
:or simply the
|
operator:numpy.logical_and
:or simply the
&
operator:If you’re using the operators then make sure you set your parenthesis correctly because of the operator precedence.
There are several logical numpy functions which should work on
pandas.Series
.The alternatives mentioned in the Exception are more suited if you encountered it when doing
if
orwhile
. I’ll shortly explain each of these:If you want to check if your Series is empty:
Python normally interprets the
len
gth of containers (likelist
,tuple
, …) as truth-value if it has no explicit boolean interpretation. So if you want the python-like check, you could do:if x.size
orif not x.empty
instead ofif x
.If your
Series
contains one and only one boolean value:If you want to check the first and only item of your Series (like
.bool()
but works even for not boolean contents):If you want to check if all or any item is not-zero, not-empty or not-False:
Well pandas use bitwise
&
|
and each condition should be wrapped in a()
For example following works
But the same query without proper brackets does not