Binning data

Now I am trying to build new SAR expansion method. And to do that, I want to convert a dataset from continues to binned one.
I searched google and found some method to achieve that. One is using Numpy and the other one is using Pandas.
I thought using pandas is more efficient way.
Basic example is following.
Numpy has digitize method. The method returns indices of the bins to which each value in input array belongs.

# using numpy
import numpy as np
x = np.array( [ 0.2, 6.4, 3.0, 1.6 ] )
bins = np.array( [ 0.0,1.0, 2.5, 4.0,10.0 ] )
inds = np.digitize( x,bins )
print( inds )
Out[91]:
 array([1, 4, 3, 2])

Next example is using Pandas.
Pandas has cut function.

cut method can handle labels argument to return results as labels.

# using pandas
import numpy as np
import pandas as pd
x = np.array( [ 0.2, 6.4, 3.0, 1.6 ] )

res=pd.cut( x,5 )
res2=pd.cut( x,5, labels=['a','b','c','d','e'] )

print( res )
Out[86]:
[(0.194, 1.44], (5.16, 6.4], (2.68, 3.92], (1.44, 2.68]]
Categories (5, object): [(0.194, 1.44] < (1.44, 2.68] < (2.68, 3.92] < (3.92, 5.16] < (5.16, 6.4]]

print( res2 )
Out[89]:
[a, e, c, b]
Categories (5, object): [a < b < c < d < e]

That’s all.

Try Grumpy.

Somedays ago, I read a news from google. Google developed “Grumpy”, it is an experimental runtime for Go.
https://opensource.googleblog.com/2017/01/grumpy-go-running-python.html
Grumpy translates python code into Go programs.
I do not have knowledge about Golang, but it seems interesting.
So, I used grumpy.
Grumpy can get from github.
https://github.com/google/grumpy
If user want to use grumpy, install is very easy.

iwatobipen$ git clone https://github.com/google/grumpy.git
iwatobipen$ grumpy

I made sample snippet.
test.py

print( "I started Grumpy!!" )
for i in range( 10 ):
    print( "GO! \n"* i )
for i in range( 20 ):
    if i % 2 == 0:
        print( i )
    else:
        print( "hoge" )

Then set environment.

iwatobipen$ make
iwatobipen$ export GOPATH=$PWD/build
iwatobipen$ export PYTHONPATH=$PWD/build/lib/python2.7/site-packages

Then compile the python script and build a binary from it.

iwatobipen$ tools/grumpc test.py > test.go
iwatobipen$ go build test.go

Run test.

iwatobipen$ time python test.py 
I started Grumpy!!
......
real	0m0.102s
user	0m0.042s
sys	0m0.026s

iwatobipen$ time go run test.go 
I started Grumpy!!
.....
real	0m1.288s
user	0m1.397s
sys	0m0.211s

iwatobipen$ time ./test
I started Grumpy!!
....
real	0m0.015s
user	0m0.008s
sys	0m0.007s

It works.