Does not work with generator functions?

It appears that memory_profiler does not produce any output at all if asked to profile a generator function (I couldn't find this documented anywhere, so I assume it's a bug)..

I took the simple example code snippet from https://pypi.python.org/pypi/memory_profiler and saved it as example.py. I then took a copy and modified it as follows and saved it to example2.py:

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    yield a

if __name__ == '__main__':
    next(my_func())

(i.e. replaced the "return" with a "yield" instead). I got the following results:

$ python3 -m memory_profiler example.py 
Filename: example.py

Line #    Mem usage    Increment   Line Contents
================================================
     1                             @profile
     2     8.969 MB     0.000 MB   def my_func():
     3    16.699 MB     7.730 MB       a = [1] * (10 ** 6)
     4   169.324 MB   152.625 MB       b = [2] * (2 * 10 ** 7)
     5    16.738 MB  -152.586 MB       del b
     6    16.738 MB     0.000 MB       return a


$ python3 -m memory_profiler example2.py 
$

yes that's a bug. Thanks for filling the issue.

Test issue pythonprofilers#42

I still see that it does not work for generator functions (version 0.41 from pip). Can it be fixed?

It should work, there are even some tests for this case. What is the error / failure?

Consider the following code:

from memory_profiler import profile

@profile
def gen():
    yield 1
    yield 2
    return 3

@profile
def stuff():
    x = yield from gen()
    print('>', x)

print(list(stuff()))

Running it gives the following output (without any profiling output)

$ python -m memory_profiler test.py
> 3
[1, 2]

We don't support the new "yield from" syntax unfortunately. Patches welcome
:-)

(BTW don't import profile if you run with "-m memory_profiler", it can have
bad side effects)

On Tue, Jul 19, 2016 at 3:20 PM, jonasc notifications@github.com wrote:

Consider the following code:

from memory_profiler import profile
@profiledef gen():
yield 1
yield 2
return 3
@profiledef stuff():
x = yield from gen()
print('>', x)
print(list(stuff()))

Running it gives the following output (without any profiling output)

$ python -m memory_profiler test.py

3
[1, 2]

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fabianp/memory_profiler/issues/42#issuecomment-233629778,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAQ8h5EkTaKuk-KmyE16QxPwcaV94LGPks5qXM8WgaJpZM4AjmU_
.

Change the title to "does not work with coroutines"? :)

what am I dong wrong?

$> cat script2.py
import memory_profiler as mp
print(mp.__version__)
from memory_profiler import profile

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    yield b
    del b
    yield a

for f in my_func():
   print('.')

$> python -m memory_profiler script2.py
0.52.0
.
.

Any progress on that?

#255

Still see this problem with generators (not coroutines) both in py2 and py3 in my code. profile wrapper just outputs nothing

E.g. script

from memory_profiler import profile

@profile
def gen():
    for i in range(100):
        yield i

@profile
def func1():
    return [i for i in gen()]

@profile
def func2():
    return list(range(100))

assert func1() == func2()

outputs for func2 and func1but nothing for gen

The problem persistis, python 3.11.

Navigation Menu

Repository navigation

Does not work with generator functions? #42

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Does not work with generator functions? #42

Description

Activity

fabianp commented on Apr 6, 2013

jonasc commented on Jul 19, 2016

fabianp commented on Jul 19, 2016

jonasc commented on Jul 19, 2016

fabianp commented on Jul 19, 2016

astrojuanlu commented on Oct 19, 2017

yarikoptic commented on Jun 7, 2018

wowkin2 commented on Jul 28, 2018

delijati commented on Apr 15, 2020

Templarrr commented on May 28, 2021

luizoti commented on Dec 12, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions