Skip to content

allow different batch sizes #5

@pilosoposerio

Description

@pilosoposerio

this is for an enhancement.

aside from setting a specific batch size for all splits, let users set a batch size per split file that will be generated.

revision in the main code can be something like this:

try:
	page = 0 # nth fragment
	origBatch = batch

	while True:
		data: List[Dict[str, Any]] = []

		batch = origBatch
		
		if batches is not None and page < len(batches):
			batch = batches[page]
			page += 1

		for _ in range(batch):
			data.append(next(features))
		
		yield geojson.FeatureCollection(data)
except StopIteration:

originally, my aim is to have a file size limit per split... but this can do for now.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions