PR#529: Reduce memory usage of images to rebuild -

freshmaker

#529 Reduce memory usage of images to rebuild

Merged Jun 9, 2020 by gnaponie. Opened May 25, 2020 by apaplaus.

apaplaus/freshmaker freshmaker_memory_leak into master

apaplaus commented May 25, 2020

When list of lists with images to be rebuilt is constructed, we will
sometimes call deduplication function on it. Because we need to reduce
its size during construction, not after it.

For start I've chosen to do deduplication every 50 new tuples of images,
because deduplication doesn't reduce amount of tuples,
it only replaces images in this tuples with newer ones.

RESOLVES: CLOUDWF-1612

Signed-off-by: Andrei Paplauski apaplaus@redhat.com

qwan commented Jun 1, 2020

:thumbsup: I think it doesn't hurt to add this, if there are some images can be deduplicated early, this should helps on reducing the memory usage of to_rebuild.

rebased onto 2b760b7d0209db3fca57324ace7a5eed2fb7c22f

Jun 4, 2020

apaplaus commented Jun 4, 2020

I've added other approach to resolve this problem, now we will not query Lightblue for all information about rpms inside container image, but will query only for necessary fields of rpms.

qwan commented Jun 8, 2020

:+1: can you update commit message of 2b760b7 to remove the 'WIP'?

rebased onto 9abcdb8f0dd363d42383d5d2f7621c7ce9e8582d

Jun 8, 2020

apaplaus commented Jun 8, 2020