`@observe x ~ dist`

[penelopeysm]

This sort of model doesn't work "as intended" (one of the oldest remaining issues, #519):

@model function f()
    x ~ Normal()
    y = x + 1
    y ~ Normal()
end

y will be treated as a random variable here.

I think it's fine to say that this is how DynamicPPL works, but the question is what workarounds for it do we have right now. The main solution is to use @addlogprob!

@model function f()
    x ~ Normal()
    y = x + 1
    @addlogprob! logpdf(Normal(), y)
end

While this works correctly for log-density tracking purposes, it's not semantically equivalent to a true observation y ~ Normal(). In particular, @addlogprob! only triggers accumulation for the log-likelihood accumulator, and any other accumulators are ignored. That means that any custom behaviour in accumulate_observe!! will not work.

For most accumulators in DynamicPPL, we actually define accumulate_observe!!(acc, ...) = acc i.e., the accumulators don't track observed variables. However, there are a couple of exceptions. One is pointwise_loglikelihoods, which does in fact hook into accumulate_observe!!. That means that for the model above, we can't extract any loglikelihoods:

pointwise_loglikelihoods(f(), VarInfo(f()))
# OrderedDict{VarName, Float64}()

whereas with a 'true' observe statement, we can:

@model function g(y)
    x ~ Normal()
    y ~ Normal()
end
pointwise_loglikelihoods(g(1.0), VarInfo(g(1.0)))
# OrderedDict{VarName, Float64} with 1 entry:
#   y => -1.41894

I think that the right solution to this is that we need something to be able to force a tilde-statement to be treated as an observation. In essence, @addlogprob! expands to just a call to accumulate_assume!! on one single accumulator, but we should have something that expands to DynamicPPL.tilde_observe!!(...) which will in turn behave like a true observation.

I threw this together (it took me way longer than I would like to admit):

macro observe(expr)
    return _observe(expr)
end
function _observe(expr)
    @gensym dist left_val val
    if Meta.isexpr(expr, :call) && length(expr.args) == 3 && expr.args[1] == :~
        left_arg, right_arg = expr.args[2], expr.args[3]
        vn = AbstractPPL.varname(left_arg, false)
        return esc(quote
            $left_val = $left_arg
            $dist = $right_arg
            $val, __varinfo__ = $(DynamicPPL.tilde_observe!!)(__model__.context, $dist, $left_val, $vn, __varinfo__)
            $val
        end)
    else
        error("@observe expects an expression of the form `var ~ dist`.")
    end
end

@model function fnew()
    x ~ Normal()
    y = x + 1
    @observe y ~ Normal()
end

and now you get

julia> pointwise_loglikelihoods(fnew(), VarInfo(fnew()))
OrderedDict{VarName, Float64} with 1 entry:
  y => -1.52277

[penelopeysm]

Actually there is another solution that already works now

@model function inner(y)
    y ~ Normal()
end
@model function fsub()
    x ~ Normal()
    y = x + 1
    _n ~ to_submodel(inner(y))
end

and has identical performance to the macro above (note that I'm testing on the unreleased 0.40 branch; I don't know what the performance on 0.39 is but I don't really care). Although that gives a useless prefix with pointwise_loglikelihoods

julia> m2 = fsub(); pointwise_loglikelihoods(m2, VarInfo(m2))
OrderedDict{VarName, Float64} with 1 entry:
  _n.y => -0.942984

[yebai]

Addressing this ambiguity cleanly and robustly is the main motivation for #965, which would prohibit intermediate variables on the LHS of the tilde.

Solving #965 also helps with several other things:

• performance, since we can preallocate all variables
• interact better with autodiff, thus removing the need for placeholder parametric types, eg, function gdemo(x, ::Type{T} = Vector{Float64}) where {T}
• more compatible with programmable inference, since all parameters are stored in an explicit argument, which is easier to work with.