Error caused when torch.compile() and torchao.autoquant() used.

[ankit-vaidya19]

torch.compile() and torchao.autoquant() work separately. But when used together, the following error is caused.

torch - 2.5.1 + cu124
torchao - 0.8.0 + cu124
diffusers - 0.33.0.dev0
GPU - A100 80GB

Code -

from diffusers import FluxPipeline
from torchao.quantization import autoquant
import torch 

pipeline = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16
).to("cuda")
pipeline.transformer.to(memory_format=torch.channels_last)
pipeline.transformer = autoquant(torch.compile(pipeline.transformer, mode="max-autotune", fullgraph=True),error_on_unseen=False)
image = pipeline(
    "a dog surfing on moon", guidance_scale=3.5, num_inference_steps=50
).images[0]

Error -

  0%|                                                                                                       | 0/50 [02:00<?, ?it/s]
Traceback (most recent call last):                                                                                                 
  File "/flux_dev.py", line 10, in <module>                                              
    image = pipeline(                                                                                                              
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in d
ecorate_context                                                                                                                    
    return func(*args, **kwargs)                                                                                                   
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py",
 line 912, in __call__                                                                                                             
    noise_pred = self.transformer(                                                                                                 
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in 
_wrapped_call_impl                                                                                                                 
    return self._call_impl(*args, **kwargs)                                                                                        
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1844, in 
_call_impl                                                                                                                         
    return inner()    
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1790, in 
inner                                                                                                                              
    result = forward_call(*args, **kwargs)                                                                                         
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 465, in 
_fn                                                                                                                                
    return fn(*args, **kwargs)                                                                                                     
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in 
_wrapped_call_impl                                                                                                                 
    return self._call_impl(*args, **kwargs)                                                                                        
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in 
_call_impl                                                                                                                         
    return forward_call(*args, **kwargs)                                                                                           
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1269,
 in __call__                                                                                                                       
    return self._torchdynamo_orig_callable(                                                                                        
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 526, 
in __call__                                                                                                                        
    return _compile(                                                                                                               
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 952, 
in _compile                                                                                                                        
    raise InternalTorchDynamoError(                                                                                                
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 924, 
in _compile                                                                                                                        
    guarded_code = compile_inner(code, one_graph, hooks, transform)                                                                
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 666, 
in compile_inner                                                                                                                   
    return _compile_inner(code, one_graph, hooks, transform)                                                                       
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrap
per_function                                                                                                                       
    return function(*args, **kwargs)                                                                                               
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 699, 
in _compile_inner                                                                                                                  
    out_code = transform_code_object(code, transform)                                                                              
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", 
line 1322, in transform_code_object                                                                                                
    transformations(instructions, code_options)                                                                                    
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 219, 
in _fn                                                                                                                             
    return fn(*args, **kwargs)                                                                                                     
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 634, 
in transform                                                                                                                       
    tracer.run()                                                                                                                   
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 27
96, in run                                                                                                                         
    super().run()                                                                                                                   
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 98
3, in run                                                                                                                          
    while self.step():                                                                                                             
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 89
5, in step                                                                                                                         
    self.dispatch_table[inst.opcode](self, inst)                                                                                   
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 29
87, in RETURN_VALUE                                                                                                                
    self._return(inst)                                                                                                             
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 29
72, in _return                                                                                                                     
    self.output.compile_subgraph(                                                                                                  
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1142, 
in compile_subgraph                                                                                                                
    self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)                                                            
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1318, 
in compile_and_call_fx_graph                                                                                                       
    fx.GraphModule(root, self.graph),                                                                                              
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/fx/graph_module.py", line 471, in __i
nit__                                                                                                                              
    self.graph = graph                                                                                                             
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2032, in 
__setattr__                                                                                                                        
    super().__setattr__(name, value)                                                                                               
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/fx/graph_module.py", line 518, in gra
ph                                                                                                                                 
    self.recompile()                                                                                                               
  File /anaconda3/envs/optim/lib/python3.10/site-packages/torch/fx/graph_module.py", line 770, in rec
ompile                                                                                                                             
    cls.forward = _forward_from_src(self._code, python_code.globals, co_fields)                                                    
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 178, 
in fx_forward_from_src_skip_result                                                                                                 
    result = original_forward_from_src(src, globals, co_fields)                                                                    
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/fx/graph_module.py", line 90, in _for
ward_from_src                                                                                                                      
    return _method_from_src(                                                                                                       
  File "/anaconda3/envs/optim/lib/python3.10/site-packages/torch/fx/graph_module.py", line 100, in _me
thod_from_src                                                                                                                      
    _exec_with_source(src, globals_copy, co_fields)                                                                                
  File "anaconda3/envs/optim/lib/python3.10/site-packages/torch/fx/graph_module.py", line 86, in _exe
c_with_source                                                                                                                      
    exec(compile(src, key, "exec"), globals)                                                                                       
torch._dynamo.exc.InternalTorchDynamoError: SystemError: excessive stack use: stack is 6202 deep 

[sayakpaul]

Can you try with a PyTorch nightly?

[ankit-vaidya19]

The same error persists with the current torch nighly build

[sayakpaul]

How about this?

from diffusers import DiffusionPipeline
import torch 
from torchao.quantization import autoquant

pipeline = DiffusionPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16,
)

pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda")
pipeline.transformer.to(memory_format=torch.channels_last)
pipeline.transformer = torch.compile(pipeline.transformer, mode="max-autotune", fullgraph=True)
pipeline.transformer = autoquant(pipeline.transformer, error_on_unseen=False)

image = pipeline(
        prompt="a cat sitting by the sea waiting for its companion to come", 
    guidance_scale=3.5, 
    num_inference_steps=28, 
    max_sequence_length=512,
    generator=torch.manual_seed(0)
).images[0]

[takhyun12]

I have the same issue.

I’m encountering an issue whenever I try to combine PyTorch’s torch.compile() with torchao autoquant in the same pipeline. Each feature alone works fine, but when used together (regardless of the order: compile → autoquant or autoquant → compile), I get a runtime failure related to FakeTensor not having an int_data attribute.

Below is my system information and a snippet of the error log. It seems like a conflict between PyTorch Dynamo’s FakeTensor mechanism and the custom int8 logic in autoquant (e.g., w_qtensor.tensor_impl.int_data). Any guidance or a workaround to make both torch.compile() and autoquant co-exist without error would be appreciated.

Environment:

Item	Specification
OS	Ubuntu 20.04.6 LTS
CPU	AMD EPYC 7543 32-Core Processor
GPU	NVIDIA A100 80GB
NVIDIA Driver Version	535.129.03
CUDA Version	12.2

Dependencies:

torch==2.4.1
diffusers==0.32.2
transformers==4.46.3
accelerate==1.4.0
protobuf==5.29.3
sentencepiece==0.2.0
peft==0.14.0
torchao==0.8.0

Error log:

activation_shapes: torch.Size([7854, 3072]), times_seen: 1
weight_shape: torch.Size([64, 3072]), dtype: torch.bfloat16, bias_shape: torch.Size([64])
>>time: 0.060ms for <class 'torchao.quantization.autoquant.AQDefaultLinearWeight'>, to_beat: infms 
>>time: 0.066ms for <class 'torchao.quantization.autoquant.AQInt8WeightOnlyQuantizedLinearWeight'>, to_beat: 0.060ms 
>>time: 0.043ms for <class 'torchao.quantization.autoquant.AQInt8DynamicallyQuantizedLinearWeight'> matmul, to_beat: 0.060ms
/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:663: UserWarning: Graph break due to unsupported builtin None.TensorBase._make_wrapper_subclass. This function is either a Python builtin (e.g. _warnings.warn)
 or a third-party C/C++ Python extension (perhaps created with pybind). If it is a Python builtin, please file an issue on GitHub so the PyTorch team can add support for it and see the next case for a workaround. If it is a third-party C/C++ Python extension, please either wrap it into a PyTorch-understood custom operator (see https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html for more details) or, if it is traceable, use torch.compiler.allow_in_graph.
  torch._dynamo.utils.warn_once(msg)
>>time: 0.170ms for <class 'torchao.quantization.autoquant.AQInt8DynamicallyQuantizedLinearWeight'>, to_beat: 0.159ms 
>>time: 0.151ms for <class 'torchao.quantization.autoquant.AQInt8DynamicallyQuantizedLinearWeight'> interpolated, breakeven constant: 0.14
best_cls=<class 'torchao.quantization.autoquant.AQDefaultLinearWeight'>

  0%|                                                                                                                                                                                                                          | 0/50 [03:25<?, ?it/s]
Traceback (most recent call last):
  File "/root/snowflake/experimental/flux/main.py", line 47, in <module>
    output_image = pipeline(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux_fill.py", line 916, in __call__
    noise_pred = self.transformer(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1603, in _call_impl
    result = forward_call(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 433, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1116, in __call__
    return self._torchdynamo_orig_callable(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 472, in __call__
    return _compile(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_utils_internal.py", line 84, in wrapper_function
    return StrobelightCompileTimeProfiler.profile_compile_time(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time
    return func(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1185, in transform_code_object
    transformations(instructions, code_options)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 582, in transform
    tracer.run()
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2451, in run
    super().run()
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
    while self.step():
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 499, in wrapper
    return inner_fn(self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1459, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 437, in call_function
    return tx.inline_user_function_return(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 749, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2666, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2782, in inline_call_
    tracer.run()
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
    while self.step():
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 499, in wrapper
    return inner_fn(self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1500, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 344, in call_function
    return super().call_function(tx, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
    return super().call_function(tx, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
    return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 749, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2666, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2782, in inline_call_
    tracer.run()
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
    while self.step():
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 499, in wrapper
    return inner_fn(self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1459, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 437, in call_function
    return tx.inline_user_function_return(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 749, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2666, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2782, in inline_call_
    tracer.run()
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
    while self.step():
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 499, in wrapper
    return inner_fn(self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1500, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 344, in call_function
    return super().call_function(tx, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
    return super().call_function(tx, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
    return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 749, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2666, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2782, in inline_call_
    tracer.run()
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
    while self.step():
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 499, in wrapper
    return inner_fn(self, inst)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1459, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 409, in call_function
    return wrap_fx_proxy(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1713, in wrap_fx_proxy
    return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1798, in wrap_fx_proxy_cls
    example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1853, in get_fake_value
    raise TorchRuntimeError(str(e)).with_traceback(e.__traceback__) from None
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1785, in get_fake_value
    ret_val = wrap_fake_exception(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1300, in wrap_fake_exception
    return fn()
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1786, in <lambda>
    lambda: run_node(tx.output, node, args, kwargs, nnmodule)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1921, in run_node
    raise RuntimeError(make_error_message(e)).with_traceback(
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1908, in run_node
    return nnmodule(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 117, in forward
    return F.linear(input, self.weight, self.bias)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torchao/utils.py", line 433, in _dispatch__torch_function__
    return cls._ATEN_OP_OR_TORCH_FN_TABLE[func](func, types, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torchao/utils.py", line 412, in wrapper
    return func(f, types, args, kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torchao/dtypes/affine_quantized_tensor_ops.py", line 179, in _
    return weight_tensor._quantized_linear_op(input_tensor, weight_tensor, bias)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torchao/quantization/autoquant.py", line 578, in _quantized_linear_op
    y = (act_mat * w_qtensor.tensor_impl.int_data.t().unsqueeze(0)).sum(dim=-2)
torch._dynamo.exc.TorchRuntimeError: Failed running call_module L__self___time_text_embed_timestep_embedder_linear_1(*(FakeTensor(..., device='cuda:0', size=(1, 256), dtype=torch.bfloat16),), **{}):
'FakeTensor' object has no attribute 'int_data'

from user code:
   File "/opt/conda/envs/flux/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 474, in forward
    else self.time_text_embed(timestep, guidance, pooled_projections)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/diffusers/models/embeddings.py", line 1621, in forward
    timesteps_emb = self.timestep_embedder(timesteps_proj.to(dtype=pooled_projection.dtype))  # (N, D)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/flux/lib/python3.10/site-packages/diffusers/models/embeddings.py", line 1304, in forward
    sample = self.linear_1(sample)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

[jerryzh168]

torch==2.4.1

This error is probably caused by torch version, it's too old I think, could you reinstall the latest torch and try again