Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
1e96fff
Fix scaling bug that can occur when some bin stats are None
jbeilstenedmands May 31, 2019
4fb77c8
Clean clutter
DiamondLightSource-build-server Jun 2, 2019
8914e88
Remove option to specify space group in dials.scale
jbeilstenedmands May 31, 2019
a3c3efb
Remove custom pytest marker from dials.scale tests
jbeilstenedmands Jun 3, 2019
84bfad9
Remove setpaths* from installer
Anthchirp Jun 4, 2019
0a3422a
This is true by default.
Anthchirp Jun 4, 2019
83a7855
Restore reflection selection in reciprocal_lattice_viewer
ndevenish Jun 5, 2019
61ad42a
Don't set matplotlib backend in per_image_analysis
ndevenish Jun 3, 2019
917d3a0
Flake8 and tidy per_image_analysis.py
ndevenish Jun 3, 2019
0d9ed72
RLV: imagesets -> experiments
Jun 5, 2019
26440c6
Replace some Sorry exceptions in scaling.
jbeilstenedmands Jun 5, 2019
7383e9a
Replace Sorry calls with ValueError where appropriate.
jbeilstenedmands Jun 5, 2019
003b64b
Add .flake8 file for manual validation
Anthchirp Jun 6, 2019
01921d5
Fix dials.rl_png command
Anthchirp Jun 6, 2019
80b97ce
fix spacing in error message
Anthchirp Jun 6, 2019
22c7c4c
Fix assign_identifiers bug introduced in #7383e9a
jbeilstenedmands Jun 6, 2019
5f2e6fc
Fix dials.export format=json
Anthchirp Jun 6, 2019
48205da
Remove use of sorry from non-cmd export code.
jbeilstenedmands Jun 6, 2019
f809cd5
precommitbx: build libffi
Anthchirp Jun 6, 2019
8718ebd
fix broken test logic
Jun 6, 2019
033c4d5
refactor basis vector search code into strategies
Feb 19, 2019
c720abf
refactor basis vector combination code
Feb 19, 2019
aca96f6
use basis vector search strategies
Feb 19, 2019
73e9358
use refactored basis vector combination code
Feb 19, 2019
02af1ac
skip these tests as cluster_analysis_search=True not currently implem…
Feb 19, 2019
dfe9394
minor relaxation of tolerance
Feb 19, 2019
bad0da1
refactor assign_indices to strategy pattern; move to separate file; m…
Feb 22, 2019
60fc2b5
find_basis_vectors also returns used_in_indexing selection: enables i…
Feb 22, 2019
b7c4884
move filtering of similar unit cell orientations to outside of choose…
Feb 25, 2019
34b9b4f
missing import
Feb 25, 2019
60e54a8
move non-primitive basis code to separate module; add tests
Mar 21, 2019
d7c305a
use new location for non_primitive_basis.correct()
Mar 21, 2019
402a530
move/rename SolutionTrackers to new model_evaluation module
Mar 21, 2019
b494cc2
move run_one_refinement inner function to new class in model_evaluation
Mar 21, 2019
7286158
pytest.approx
Mar 21, 2019
b323b6b
this code now lives in model_evaluation
Mar 22, 2019
997aa89
Attempt to encapsulate handling of known symmetry into external class
Mar 29, 2019
c617e1d
Consolidate fft1d, fft3d, rsgs classes into single BasisVectorSearch …
Mar 29, 2019
6797000
Add test for stills indexer; fix stills indexer
Mar 29, 2019
d3a44c2
this code has already been moved elsewhere
Mar 29, 2019
1019810
Add reflection_table.calculate_entering_flags()
Apr 1, 2019
9c97907
Move basis vector optimisation code to new module.
Apr 1, 2019
8a9aea7
remove unused functions
Apr 1, 2019
018d246
Refactor dials.index command line script.
Apr 1, 2019
48c0f4d
reinstate max_combinations parameter
Apr 1, 2019
6cae3ad
Move phil definitions closer to code where they are used
Apr 1, 2019
716078b
scan_range -> image_range
Apr 23, 2019
a29a1eb
remove non-existing import
Apr 23, 2019
96b33e7
pickle -> msgpack
Apr 23, 2019
35a8149
fix test
Apr 23, 2019
2e77748
move all indexing tests to algorithms/indexing
Apr 23, 2019
6a154c8
remove deprecated show_rotation_matrix_differences() function
Apr 24, 2019
3bde77c
add test for rotation_matrix_differences() function
Apr 24, 2019
6ae68eb
Remove commented code; comments-> docstring
Apr 24, 2019
3ac49e3
improve code coverage of tests
Apr 24, 2019
5dc7bf2
remove dead code
Apr 24, 2019
c7bac78
fix import after moving test location
Apr 24, 2019
a4f4dee
soften tests for optimise_basis_vectors
Apr 24, 2019
95b1ad7
remove debug_plots functionality
Apr 24, 2019
5bbb611
improve code coverage of max_cell tests
Apr 24, 2019
d56034e
remove debug_plot_candidate_basis_vectors() function
Apr 24, 2019
9190a06
add simple test for BasisVectorSearch class
Apr 24, 2019
f6f9b8a
start refactoring of indexer.index() function
May 7, 2019
80e2c3a
Add stills indexer BasisVectorSearch test
May 8, 2019
a1efd04
these files should have been removed with fed639b39345782ff833ba89fb9…
May 8, 2019
0e36404
move filter_similar_orientations to standalone function; add test
May 8, 2019
fee1ab7
fix tests
May 8, 2019
20a2b96
docstrings for strategies module
May 8, 2019
b2392b8
search_beam_position: scan_range -> image_range; add test
May 9, 2019
cdb94f7
add option to index multiple experimemts independently (joint_indexin…
May 15, 2019
a34e3d0
mark method as private
May 16, 2019
4cab6a7
updated index api docs
May 16, 2019
817220d
test for indexing multiple experiments with joint_indexing=False
May 31, 2019
0d0b1d0
don't use locals()
May 31, 2019
500856d
missing import; add test
Jun 4, 2019
039208e
I don't think this method is called
Jun 4, 2019
c772c78
address flake8 unassigned local variable warning (F841)
Jun 4, 2019
3822f0d
fix test_compare_orientation_matrices
Jun 4, 2019
d597488
fix test_phi_scan indavertantly broken by 4f9f801fdbfcbac89197ecfa129…
Jun 4, 2019
b47442a
Add missed 'Sorry' imports
ndevenish Jun 3, 2019
0991b18
replace Sorry with custom DialsIndexError
Jun 5, 2019
f4bfb12
loosen test tolerance
Jun 5, 2019
d5ca0a9
catch DialsIndexError and re-raise as Sorry
Jun 5, 2019
7cc8b85
Parallel indexing of multiple experiments
Jun 5, 2019
dd5454a
PEP8 class naming convention
Jun 5, 2019
0c791d9
pacify flake8
Jun 5, 2019
5e6a104
tidy imports
Jun 5, 2019
3dda7f9
I don't think this is required any more
Jun 5, 2019
ac93bdc
fix some lgtm issues
Jun 5, 2019
7d6a659
>= not >
Jun 5, 2019
febc596
add deprecated index_reflections() function for backwards compatibility
Jun 6, 2019
324a522
this break statement can never be hit
Jun 6, 2019
629ce67
remove unreachable code
Jun 6, 2019
b40d54f
xrange -> range
Jun 6, 2019
bd07aec
perhaps this isn't needed after all?
Jun 6, 2019
b36aa93
missed off unindexed reflections (removed by 667b8c8e985e7b5b9bbf167e…
Jun 6, 2019
ca8a883
fix find_spots_server
Jun 6, 2019
0de7dd8
fix flake8 issues
Jun 6, 2019
b65ed45
remove unused variable
Jun 6, 2019
22abf6e
don't convert generator to list as might not need all items
Jun 7, 2019
50638a9
print -> logging
Jun 7, 2019
aeb31e5
implement basis vector search strategies as entry points
Jun 7, 2019
a9a0e9b
Appease the flake8 Gods
Jun 7, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Configure flake8 with some defaults to reduce noise

[flake8]
# Black disagrees with flake8 on a few points. Ignore those.
ignore = E203, E266, E501, W503
max-line-length = 88

142 changes: 16 additions & 126 deletions algorithms/indexing/__init__.py
Original file line number Diff line number Diff line change
@@ -1,138 +1,28 @@
from __future__ import absolute_import, division, print_function

from cctbx.array_family import flex
from dials_algorithms_indexing_ext import *
import warnings

import dials_algorithms_indexing_ext as ext

def index_reflections(reflections, experiments, d_min=None, tolerance=0.3):
reciprocal_lattice_points = reflections["rlp"]
reflections["miller_index"] = flex.miller_index(len(reflections), (0, 0, 0))
if d_min is not None:
d_spacings = 1 / reciprocal_lattice_points.norms()
inside_resolution_limit = d_spacings > d_min
else:
inside_resolution_limit = flex.bool(reciprocal_lattice_points.size(), True)
sel = inside_resolution_limit & (reflections["id"] == -1)
isel = sel.iselection()
rlps = reciprocal_lattice_points.select(isel)
refs = reflections.select(isel)
phi = refs["xyzobs.mm.value"].parts()[2]

UB_matrices = flex.mat3_double([cm.get_A() for cm in experiments.crystals()])
imgset_ids = reflections["imageset_id"].select(sel)

for i_imgset, imgset in enumerate(experiments.imagesets()):
sel_imgset = imgset_ids == i_imgset

result = AssignIndices(
rlps.select(sel_imgset),
phi.select(sel_imgset),
UB_matrices,
tolerance=tolerance,
)

miller_indices = result.miller_indices()
crystal_ids = result.crystal_ids()
sampling_volume_map = ext.sampling_volume_map

expt_ids = flex.int(crystal_ids.size(), -1)
for i_cryst, cryst in enumerate(experiments.crystals()):
sel_cryst = crystal_ids == i_cryst
for i_expt in experiments.where(crystal=cryst, imageset=imgset):
expt_ids.set_selected(sel_cryst, i_expt)

reflections["miller_index"].set_selected(
isel.select(sel_imgset), miller_indices
)
reflections["id"].set_selected(isel.select(sel_imgset), expt_ids)
reflections.set_flags(
reflections["miller_index"] != (0, 0, 0), reflections.flags.indexed
)
reflections["id"].set_selected(reflections["miller_index"] == (0, 0, 0), -1)
class DialsIndexError(RuntimeError):
pass


def index_reflections_local(
reflections,
experiments,
d_min=None,
epsilon=0.05,
delta=8,
l_min=0.8,
nearest_neighbours=20,
):
from scitbx import matrix
from libtbx.math_utils import nearest_integer as nint
class DialsIndexRefineError(DialsIndexError):
pass

reciprocal_lattice_points = reflections["rlp"]
if "miller_index" not in reflections:
reflections["miller_index"] = flex.miller_index(len(reflections))
if d_min is not None:
d_spacings = 1 / reciprocal_lattice_points.norms()
inside_resolution_limit = d_spacings > d_min
else:
inside_resolution_limit = flex.bool(reciprocal_lattice_points.size(), True)
sel = inside_resolution_limit & (reflections["id"] == -1)
isel = sel.iselection()
rlps = reciprocal_lattice_points.select(isel)
refs = reflections.select(isel)
phi = refs["xyzobs.mm.value"].parts()[2]

if len(rlps) <= nearest_neighbours:
from dials.util import Sorry

raise Sorry(
"index_assignment.local.nearest_neighbour must be smaller than the number of accepted reflections (%d)"
% len(rlps)
)

UB_matrices = flex.mat3_double([cm.get_A() for cm in experiments.crystals()])

result = AssignIndicesLocal(
rlps,
phi,
UB_matrices,
epsilon=epsilon,
delta=delta,
l_min=l_min,
nearest_neighbours=nearest_neighbours,
)
miller_indices = result.miller_indices()
crystal_ids = result.crystal_ids()
hkl = miller_indices.as_vec3_double().iround()

assert miller_indices.select(crystal_ids < 0).all_eq((0, 0, 0))

for i_cryst in set(crystal_ids):
if i_cryst < 0:
continue

A = matrix.sqr(experiments[i_cryst].crystal.get_A())
A_inv = A.inverse()

cryst_sel = crystal_ids == i_cryst
rlp_sel = rlps.select(cryst_sel)
hkl_sel = hkl.select(cryst_sel).as_vec3_double()

d_sel = 1 / rlp_sel.norms()
d_perm = flex.sort_permutation(d_sel, reverse=True)

hf_0 = A_inv * rlp_sel[d_perm[0]]
h_0 = matrix.col([nint(j) for j in hf_0.elems])
offset = h_0 - matrix.col(hkl_sel[d_perm[0]])
# print "offset:", offset.elems

h = hkl_sel + flex.vec3_double(hkl_sel.size(), offset.elems)

refs["miller_index"].set_selected(
cryst_sel, flex.miller_index(list(h.iround()))
)
refs["id"].set_selected(cryst_sel, i_cryst)

crystal_ids.set_selected(crystal_ids < 0, -1)
refs["id"] = crystal_ids
refs["miller_index"].set_selected(crystal_ids < 0, (0, 0, 0))
def index_reflections(reflections, experiments, d_min=None, tolerance=0.3):
from dials.algorithms.indexing import assign_indices

reflections["miller_index"].set_selected(isel, refs["miller_index"])
reflections["id"].set_selected(isel, refs["id"])
reflections.set_flags(
reflections["miller_index"] != (0, 0, 0), reflections.flags.indexed
warnings.warn(
"index_reflections is deprectated, use "
"dials.algorithms.indexing.assign_indices.AssignIndicesGlobal instead",
DeprecationWarning,
stacklevel=2,
)
index = assign_indices.AssignIndicesGlobal(tolerance=tolerance)
index(reflections, experiments)
159 changes: 159 additions & 0 deletions algorithms/indexing/assign_indices.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
from __future__ import absolute_import, division

import abc

from cctbx.array_family import flex
import dials_algorithms_indexing_ext as ext

from dials.algorithms.indexing import DialsIndexError


class AssignIndicesStrategy(object):

__metaclass__ = abc.ABCMeta

def __init__(self, d_min=None):
self._d_min = d_min

@abc.abstractmethod
def __call__(self, reciprocal_lattice_vectors):
pass


class AssignIndicesGlobal(AssignIndicesStrategy):
def __init__(self, tolerance=0.3):
super(AssignIndicesGlobal, self).__init__()
self._tolerance = tolerance

def __call__(self, reflections, experiments, d_min=None):
reciprocal_lattice_points = reflections["rlp"]
reflections["miller_index"] = flex.miller_index(len(reflections), (0, 0, 0))
if d_min is not None:
d_spacings = 1 / reciprocal_lattice_points.norms()
inside_resolution_limit = d_spacings > d_min
else:
inside_resolution_limit = flex.bool(reciprocal_lattice_points.size(), True)
sel = inside_resolution_limit & (reflections["id"] == -1)
isel = sel.iselection()
rlps = reciprocal_lattice_points.select(isel)
refs = reflections.select(isel)
phi = refs["xyzobs.mm.value"].parts()[2]

UB_matrices = flex.mat3_double([cm.get_A() for cm in experiments.crystals()])
imgset_ids = reflections["imageset_id"].select(sel)

for i_imgset, imgset in enumerate(experiments.imagesets()):
sel_imgset = imgset_ids == i_imgset

result = ext.AssignIndices(
rlps.select(sel_imgset),
phi.select(sel_imgset),
UB_matrices,
tolerance=self._tolerance,
)

miller_indices = result.miller_indices()
crystal_ids = result.crystal_ids()

expt_ids = flex.int(crystal_ids.size(), -1)
for i_cryst, cryst in enumerate(experiments.crystals()):
sel_cryst = crystal_ids == i_cryst
for i_expt in experiments.where(crystal=cryst, imageset=imgset):
expt_ids.set_selected(sel_cryst, i_expt)

reflections["miller_index"].set_selected(
isel.select(sel_imgset), miller_indices
)
reflections["id"].set_selected(isel.select(sel_imgset), expt_ids)
reflections.set_flags(
reflections["miller_index"] != (0, 0, 0), reflections.flags.indexed
)
reflections["id"].set_selected(reflections["miller_index"] == (0, 0, 0), -1)


class AssignIndicesLocal(AssignIndicesStrategy):
def __init__(
self, d_min=None, epsilon=0.05, delta=8, l_min=0.8, nearest_neighbours=20
):
super(AssignIndicesLocal, self).__init__()
self._epsilon = epsilon
self._delta = delta
self._l_min = l_min
self._nearest_neighbours = nearest_neighbours

def __call__(self, reflections, experiments, d_min=None):
from scitbx import matrix
from libtbx.math_utils import nearest_integer as nint

reciprocal_lattice_points = reflections["rlp"]
if "miller_index" not in reflections:
reflections["miller_index"] = flex.miller_index(len(reflections))
if d_min is not None:
d_spacings = 1 / reciprocal_lattice_points.norms()
inside_resolution_limit = d_spacings > d_min
else:
inside_resolution_limit = flex.bool(reciprocal_lattice_points.size(), True)
sel = inside_resolution_limit & (reflections["id"] == -1)
isel = sel.iselection()
rlps = reciprocal_lattice_points.select(isel)
refs = reflections.select(isel)
phi = refs["xyzobs.mm.value"].parts()[2]

if len(rlps) <= self._nearest_neighbours:
raise DialsIndexError(
"index_assignment.local.nearest_neighbour must be smaller than the number of accepted reflections (%d)"
% len(rlps)
)

UB_matrices = flex.mat3_double([cm.get_A() for cm in experiments.crystals()])

result = ext.AssignIndicesLocal(
rlps,
phi,
UB_matrices,
epsilon=self._epsilon,
delta=self._delta,
l_min=self._l_min,
nearest_neighbours=self._nearest_neighbours,
)
miller_indices = result.miller_indices()
crystal_ids = result.crystal_ids()
hkl = miller_indices.as_vec3_double().iround()

assert miller_indices.select(crystal_ids < 0).all_eq((0, 0, 0))

for i_cryst in set(crystal_ids):
if i_cryst < 0:
continue

A = matrix.sqr(experiments[i_cryst].crystal.get_A())
A_inv = A.inverse()

cryst_sel = crystal_ids == i_cryst
rlp_sel = rlps.select(cryst_sel)
hkl_sel = hkl.select(cryst_sel).as_vec3_double()

d_sel = 1 / rlp_sel.norms()
d_perm = flex.sort_permutation(d_sel, reverse=True)

hf_0 = A_inv * rlp_sel[d_perm[0]]
h_0 = matrix.col([nint(j) for j in hf_0.elems])
offset = h_0 - matrix.col(hkl_sel[d_perm[0]])
# print "offset:", offset.elems

h = hkl_sel + flex.vec3_double(hkl_sel.size(), offset.elems)

refs["miller_index"].set_selected(
cryst_sel, flex.miller_index(list(h.iround()))
)
refs["id"].set_selected(cryst_sel, i_cryst)

crystal_ids.set_selected(crystal_ids < 0, -1)
refs["id"] = crystal_ids
refs["miller_index"].set_selected(crystal_ids < 0, (0, 0, 0))

reflections["miller_index"].set_selected(isel, refs["miller_index"])
reflections["id"].set_selected(isel, refs["id"])
reflections.set_flags(
reflections["miller_index"] != (0, 0, 0), reflections.flags.indexed
)
Empty file.
Loading