Proposals of varying dimensions can't be combined

[penelopeysm]

using AdvancedMH, Distributions, LinearAlgebra

# Yes, I know this density is bogus
m = DensityModel(x -> x[1] + x[2] + x[3])
p = [StaticProposal(Normal()), RandomWalkProposal(MvNormal(zeros(2), I))]
sample(m, MetropolisHastings(p), 10; chain_type=Any, progress=false)

#=
ERROR: BoundsError: attempt to access 2-element Vector{Any} at index [3]
Stacktrace:
  [1] throw_boundserror(A::Vector{Any}, I::Tuple{Int64})
    @ Base ./essentials.jl:14
  [2] getindex
    @ ./essentials.jl:916 [inlined]
  [3] (::var"#3#4")(x::Vector{Any})
    @ Main ./REPL[16]:1
  [4] logdensity
    @ ~/ppl/amh/src/AdvancedMH.jl:74 [inlined]
  [5] transition(sampler::MetropolisHastings{Vector{AdvancedMH.Proposal}}, model::DensityModel{var"#3#4"}, params::Vector{Any}, accepted::Bool)
    @ AdvancedMH ~/ppl/amh/src/mh-core.jl:66
  [6] step(rng::Random.TaskLocalRNG, model::DensityModel{var"#3#4"}, sampler::MetropolisHastings{Vector{AdvancedMH.Proposal}}; initial_params::Nothing, kwargs::@Kwargs{num_warmup::Int64, discard_sample::Bool})
    @ AdvancedMH ~/ppl/amh/src/mh-core.jl:84
  [7] step
    @ ~/ppl/amh/src/mh-core.jl:76 [inlined]
=#

Of course, in this case, I could simply have split the RW proposal up into two sub-elements. However, that means that you can't supply a (non-diagonal) covariance matrix for the two elements as a block.

This is problematic for properly using AdvancedMH in Turing (cf. TuringLang/Turing.jl#2593) because the desired proposal above directly maps to something like:

using Turing

@model function f()
    x ~ Normal()
    y ~ Dirichlet(ones(3)))
end

where one might want to sample x with a static proposal and y with RWMH in linked space. I would like to create a LogDensityFunction from the above model with x unlinked and y linked, and then just use AdvancedMH to do the sampling, but I can't currently do it (not generally, anyway).

The only way I found to combine proposals with varying dimensions is to use a NamedTuple of proposals, but LogDensityFunction creates a vector of parameters so according to the current AdvancedMH interface the proposals also have to be a vector.

[penelopeysm]

In general other options are also broken

julia> p2 = [StaticProposal(MvNormal(zeros(2), I)), RandomWalkProposal(Normal())]
2-element Vector{AdvancedMH.Proposal}:
 StaticProposal{false, IsoNormal}(IsoNormal(
dim: 2
μ: [0.0, 0.0]
Σ: [1.0 0.0; 0.0 1.0]
)
)
 RandomWalkProposal{false, Normal{Float64}}(Normal{Float64}(μ=0.0, σ=1.0))

julia> sample(m, MetropolisHastings(p2), 10; chain_type=Any, progress=false)
ERROR: BoundsError: attempt to access 2-element Vector{Any} at index [3]

[yebai]

@sunxd3, can you help with this?