Generic Utils

These are generic utils used across the Sparkly library.

sparkly.utils.absolute_path(file_path, *rel_path)[source]

Returns absolute path to file.

Usage:
>>> absolute_path('/my/current/dir/x.txt', '..', 'x.txt')
'/my/current/x.txt'
>>> absolute_path('/my/current/dir/x.txt', 'relative', 'path')
'/my/current/dir/relative/path'
>>> import os
>>> absolute_path('x.txt', 'relative/path') == os.getcwd() + '/relative/path'
True
Parameters:
  • file_path (str) – file
  • rel_path (list[str]) – path parts
Returns:

str

sparkly.utils.kafka_get_topics_offsets(host, topic, port=9092)[source]

Returns available partitions and their offsets for the given topic.

Parameters:
  • host (str) – Kafka host.
  • topic (str) – Kafka topic.
  • port (int) – Kafka port.
Returns:

[ – [(partition, start_offset, end_offset)].

Return type:

int, int, int