unipy.utils package¶

Submodules¶

Module contents¶

Utility Objects.

This module provides a number of functions and objects for utility.

decorator¶

time_profiler – Function running time command-line profiler.
time_logger – Function running time log profiler.
job_wrapper – Command-line line dragging tool.
Infix – Function to operator translator.
infix – Functional API for Infix.

generator¶

ReusableGenerator – Reusable Generator.
re_generator – Functional API for ReusableGenerator.
split_generator – Split data by given size.
num_fromto_generator – Range number string pairs by given term.
dt_fromto_generator – Range date format string pairs by given term.
tm_fromto_generator – Range datetime format string pairs by given term.
timestamp_generator – Range timestamp string pairs by given term.

wrapper¶

multiprocessor – Functional wrapper for multiprocessing.
uprint – Print option interface within a function.

gdrive¶

gdrive_downloader – File downloader from Google Drive.
gdrive_uploader – File uploader to Google Drive.

unipy.utils.multiprocessor(func, worker=2, arg_zip=None, *args, **kwargs)[source]¶

Use multiprocessing as a function.

Just for convenience.

Parameters

func (Function) – Any function without lambda.
worker (int (default: 2)) – A number of processes.
arg_zip (zip (default: None)) – A zip instance.

Returns

A list contains results of each processes.

Return type

list

See also

multiprocessing.pool

Examples

>>> from unipy.utils.wrapper import multiprocessor
>>> alist = [1, 2, 3]
>>> blist = [-1, -2, -3]
>>> def afunc(x, y):
...     return x + y
...
>>> multiprocessor(afunc, arg_zip=zip(alist, blist))
[0, 0, 0]
>>> def bfunc(x):
...     return x + 2
...
>>> multiprocessor(bfunc, arg_zip=zip(alist))
[3, 4, 5]

unipy.utils.uprint(*args, print_ok=True, **kwargs)[source]¶

Print option interface.

This function is equal to print function but added print_ok option. This allows you to control printing in a function.

Parameters

*args (whatever print allows.) – It is same as print does.
print_ok (Boolean (default: True)) – An option whether you want to print something out or not.
arg_zip (zip (default: None)) – A zip instance.

unipy.utils.lprint(input_x, output, name=None)[source]¶

Print option interface.

This function is to stdout the shape of input layer & output layer in Deep Learning architecture.

Parameters

input_x (numpy.ndarray) – A numpy.ndarray object of input source.
output (numpy.ndarray) – A numpy.ndarray object of output target.
name (str (default: None)) – An optional name you want to print out.

unipy.utils.aprint(*arr, maxlen=None, name_list=None, decimals=None)[source]¶

Stdout the numpy.ndarray in pretty.

It prints the multiple numpy.ndarray out “Side by Side.”

Parameters

arr (numpy.ndarray) – Any arrays you want to print out.
maxlen (int (default: None)) – A length for each array to print out. It is automatically calculated in case of None.
name_list (list (default: None)) – A list contains the names of each arrays. Upper Alphabet is given in case of None.
decimals (int (default: None)) – A number to a specified number of digits to truncate.

Examples

>>> from unipy.utils.wrapper import aprint
>>> arr_x = np.array([
... [.6, .5, .1],
... [.4, .2, .8],
... ])
>>> arr_y = np.array([
... [.4, .6],
... [.7, .3,],
... ])
>>> aprint(arr_x, arr_y)
=========================================
|  A                 |    B             |
|  (2, 3)            |    (2, 2)        |
=========================================
|  [[0.6 0.5 0.1]    |    [[0.4 0.6]    |
|   [0.4 0.2 0.8]]   |     [0.7 0.3]]   |
=========================================
>>> aprint(arr_x, arr_y, name_list=['X', 'Y'])
=========================================
|  X                 |    Y             |
|  (2, 3)            |    (2, 2)        |
=========================================
|  [[0.6 0.5 0.1]    |    [[0.4 0.6]    |
|   [0.4 0.2 0.8]]   |     [0.7 0.3]]   |
=========================================
>>> aprint(arr_x, arr_y, arr_y[:1], name_list=['X', 'Y', 'Y_1'])
============================================================
|  X                 |    Y             |    Y_1           |
|  (2, 3)            |    (2, 2)        |    (1, 2)        |
============================================================
|  [[0.6 0.5 0.1]    |    [[0.4 0.6]    |    [[0.4 0.6]]   |
|   [0.4 0.2 0.8]]   |     [0.7 0.3]]   |                  |
============================================================

unipy.utils.time_profiler(func)[source]¶

Print wrapper for time profiling.

This wrapper prints out start, end and elapsed time.

Parameters: func (Function) – A function to profile.
Returns: A wrapped function.
Return type: Function

See also

functools.wraps decorator

Examples

>>> import unipy as up
>>> @up.time_profiler
... def afunc(i):
...     return len(list(range(i)))
...
>>> res = afunc(58)
(afunc) Start   : 2018-06-20 22:11:35.511374
(afunc) End     : 2018-06-20 22:11:35.511424
(afunc) Elapsed :             0:00:00.000050
>>> res
58

unipy.utils.time_logger(func)[source]¶

Logging wrapper for time profiling.

This wrapper logs start, end and elapsed time.

Parameters: func (Function) – A function to profile.
Returns: A wrapped function.
Return type: Function

See also

functools.wraps decorator

Examples

>>> import unipy as up
>>> @up.time_logger
... def afunc(i):
...     return len(list(range(i)))
...
>>> res = afunc(58)
(afunc) Start   : 2018-06-20 22:11:35.511374
(afunc) End     : 2018-06-20 22:11:35.511424
(afunc) Elapsed :             0:00:00.000050
>>> res
58

class unipy.utils.profiler(type='logging')[source]¶: Bases: object

unipy.utils.job_wrapper(func)[source]¶

Print wrapper for time profiling.

This wrapper prints out start & end line.

Parameters: func (Function) – A function to separate print-line job.
Returns: A wrapped function.
Return type: Function

See also

functools.wraps decorator

Examples

>>> import unipy as up
>>> @up.job_wrapper
... def afunc(i):
...     return len(list(range(i)))
...
>>> afunc(458)
----------- [afunc] START -----------

———– [afunc] END ———–

afunc : 0:00:00.000023

458

class unipy.utils.Infix(func)[source]¶

Bases: object

Wrapper for define an operator.

This wrapper translates a function to an operator.

Returns: A wrapped function.
Return type: Function

See also

functools.partial decorator

Examples

>>> @Infix
... def add(x, y):
...     return x + y
...
>>> 5 |add| 6
11
>>> instanceof = Infix(isinstance)
>>> 5 |instanceof| int
True

unipy.utils.infix(func)[source]¶

A functional API for Infix decorator.

Returns: A wrapped function.
Return type: Function

See also

unipy.utils.wrapper.infix

Examples

>>> @infix
... def add(x, y):
...     return x + y
...
>>> 5 |add| 6
11
>>> instanceof = infix(isinstance)
>>> 5 |instanceof| int
True

class unipy.utils.ReusableGenerator(generator)[source]¶

Bases: object

Temporary Interface to re-use generator for convenience.

Once assigned, It can be infinitely consumed **as long as an input generator remains un-exhausted.

_source¶

A source generator.

Type: generator

See also

generator itertools.tee

Examples

>>> from unipy.utils.generator import ReusableGenerator
>>> gen = (i for i in range(10))
>>> gen
<generator object <genexpr> at 0x11120ebf8>
>>> regen = ReusableGenerator(gen)
>>> regen
<unipy.utils.generator.ReusableGenerator object at 0x1061a97f0>
>>> list(regen)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(regen)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(gen)  # If the source is used, copied one will be exhausted too.
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(gen)
[]
>>> list(regen)
[]

unipy.utils.re_generator(generator)[source]¶

A functional API for unipy.ReusableGenerator.

Once assigned, It can be infinitely consumed **as long as an output generator is called at least one time.

Parameters: generator (generator) – An generator to copy. This original generator should not be used anywhere else, until the copied one consumed at least once.
Returns: A generator to be used infinitely.
Return type: generator

See also

generator itertools.tee

Examples

>>> from unipy.utils.generator import re_generator
>>> gen = (i for i in range(10))
>>> gen
<generator object <genexpr> at 0x11120ebf8>
>>> regen = copy_generator(gen)
>>> regen
<unipy.utils.generator.ReusableGenerator object at 0x1061a97f0>
>>> list(regen)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(regen)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(gen)  # Once the copied one is used, the source will be exhausted.
[]
>>> list(gen)
[]
>>> list(regen)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(regen)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

unipy.utils.split_generator(iterable, size)[source]¶

unipy.utils.num_fromto_generator(start, end, term)[source]¶

A range function yields pair chunks.

It had made for time-formatting query. It yields a tuple of (start, start+(term-1)) pair, until start > end.

Parameters: *args (int) – end or start, end[, term] It works like range function.
Yields: tuple – A tuple of (start, start+(term-1)) pair, until start > end.

See also

yield

Examples

>>> from unipy.utils.generator import num_fromto_generator
>>>
>>> query = 'BETWEEN {pre} AND {nxt};'
>>>
>>> q_list = [query.format(pre=item[0], nxt=item[1])
...           for item in num_fromto_generator(1, 100, 10)]
>>> print(q_list[0])
BETWEEN 1 AND 10;
>>> print(q_list[1])
BETWEEN 11 AND 20;

unipy.utils.dt_fromto_generator(start, end, day_term, tm_format='%Y%m%d')[source]¶

A range function yields datetime formats by pair.

It had made for time-formatting query. It yields a tuple of (start, start+(term-1)) pair, until start > end.

Parameters

start (str) – start datetime like ‘yyyymmdd’.
end (str) – start datetime like ‘yyyymmdd’.
day_term (int) – term of days.
tm_format ((default: '%Y%m%d')) – datetime format string.

Yields

tuple – A tuple of (start, start+(term-1)) pair, until start > end.

See also

yield

Examples

>>> from unipy.utils.generator import dt_fromto_generator
>>> dt_list = [item for item in
...            dt_fromto_generator('20170101','20170331', 10)]
>>> dt_list[:3]
[('20170101', '20170110'),
 ('20170111', '20170120'),
 ('20170121', '20170130')]

unipy.utils.tm_fromto_generator(start, end, day_term, tm_string=['000000', '235959'], tm_format='%Y%m%d')[source]¶

A range function yields datetime formats by pair.

It had made for time-formatting query. It yields a tuple of (start, start+(term-1)) pair, until start > end.

Parameters

start (str) – start datetime like ‘yyyymmdd’.
end (str) – start datetime like ‘yyyymmdd’.
day_term (int) – term of days.
tm_string (list (default: ['000000', '235959'])) – time strings to concatenate.
tm_format ((default: '%Y%m%d')) – datetime format string.

Yields

tuple – A tuple of (start, start+(term-1)) pair, until start > end.

See also

yield

Examples

>>> from unipy.utils.generator import tm_fromto_generator
>>> tm_list = [item for item in
...            tm_fromto_generator('20170101','20170331', 10)]
>>> tm_list[:3]
[('20170101000000', '20170110235959'),
 ('20170111000000', '20170120235959'),
 ('20170121000000', '20170130235959')]

unipy.utils.timestamp_generator(*args)[source]¶

A range function yields pair timestep strings.

It had made for time-formatting query. It yields a tuple of (start, start+(term-1)) pair, until start > end.

Parameters: *args (int) – end or start, end[, term] It works like range function.
Yields: tuple – A tuple of (start, start+(term-1)) pair, until start > end.

See also

yield

Examples

>>> from unipy.utils.generator import timestamp_generator
>>> timestamp_generator(1, 10, 2)
<generator object timestamp_generator at 0x10f519678>
>>> list(timestamp_generator(1, 14, 5))
[(1, 5), (6, 10), (11, 15)]
>>> begin, fin, period = 1, 10, 3
>>> list(timestamp_generator(begin, fin, period))
[(1, 3), (4, 6), (7, 9), (10, 12)]
>>> time_sequence = timestamp_generator(begin, fin, period)
>>> time_msg = "{start:2} to {end:2}, {term:2} days."
>>> for time in time_sequence:
... b, f = time
... print(time_msg.format(start=b, end=f, term=period))
...
 1 to  3,  3 days.
 4 to  6,  3 days.
 7 to  9,  3 days.
10 to 12,  3 days.

unipy.utils.gdrive_downloader(gdrive_url_id, pattern='*', download_path='./data')[source]¶

Download files in Google Drive.

Download files in Googel Drive to the given path.

Parameters

gdrive_url_id (str) – An URL ID of an Google Drive directory which contains files to download. https://drive.google.com/drive/folders/<google drive URL ID>.
pattern (str (default: '*')) – A pattern of regular expression to filter file in the target directory.
download_path (str (default: './data')) – A target directory to download files in given URL ID.

Returns

Nothing is returned.

Return type

None

See also

None()

Examples

>>> import unipy.util.gdrive import gdrive_downloader
>>> gdrive_path_id = '1LA5334-SZdizcFqkl4xO8Hty7w1q0e8h'
>>> up.gdrive_downloader(gdrive_path_id)

unipy.utils.gdrive_uploader(gdrive_url_id, pattern='*', src_dir='./data')[source]¶

Download files in Google Drive.

Download files in Googel Drive to the given path.

Parameters

gdrive_url_id (str) – An URL ID of an Google Drive directory to upload files. https://drive.google.com/drive/folders/<google drive URL ID>.
pattern (str (default: '*')) – A pattern of regular expression to filter file in the target directory.
src_dir (str (default: './data')) – A source directory to upload files in given URL ID.

Returns

Nothing is returned.

Return type

None

See also

None()

Examples

>>> import unipy.util.gdrive import gdrive_uploader
>>> gdrive_path_id = '1LA5334-SZdizcFqkl4xO8Hty7w1q0e8h'
>>> up.gdrive_uploader(gdrive_path_id)