0.16.0

Release Notes

New Features

• Support the Microsoft Purview GraphQL endpoint ( #265 )
• Support providing info values for experts / owners in Microsoft Purview when parsing the Excel bulk entities tab. ( #264 )
  • Achieved by following the same pattern as Microsoft Purview follows in their term upload which is emailaddress:info value
  • Where info value is the string of text you want in the info field.
• Added two update_entity_tags and delete_entity_tags to the PurviewClient ( #245 )
  • These are thin wrappers over update_entity_labels and delete_entity_labels to reduce the confusion for MS Purview users.
  • MS Purview calls labels as tags in the MS Purview UI.
• AtlasClient.addRelationship now supports adding a list of Atlas Entities that will automatically be converted to the minimum JSON necessary for use in the /entity and /entity/bulk endpoints.

Bug Fixes

• Fixed issue with experts and owners parsing from Excel (#266, #257, #252)

Breaking Changes

• None

Other Changes

• None

Full Changelog: 0.15.0...0.16.0

0.15.0

Release Notes

New Features

• Support Business / Managed Attributes in Excel.
  • You can include them with a column header of [Business][typeName] attributeName or
    [Managed][groupName] attributeName. Both are valid since Atlas would call it BusinessMetadata
    but Purview calls it Managed attributes
• Support Updates to Glossary Term

Bug Fixes

• get_term when get_glossary would error out if returning an empty result

Breaking Changes

• None

Other Changes

• Removed support for Python 3.6

Full Changelog: 0.14.0...0.15.0

0.14.0

Release Notes

This release mainly improves things internally and makes the package easier to manage and explore.

New Features

• Support Delete an Entity by Unique Attributes by @wjohnson in #218
• Supports create_or_update_collection in the catalog dataplane with PurviewClient.collections

Bug Fixes

• None

Breaking Changes

• None

Other Changes

• Update user agent to include "pyapacheatlas" by @sonnyhcl in #214
• Update from legacy Purview API endpoints and api versions by @wjohnson in #217
• Updated docs to use pydata theme and structure by @wjohnson in #219
• Cleaning up code for flake8 by @wjohnson in #220

New Contributors

• @sonnyhcl made their first contribution in #214

Full Changelog: 0.13.0...0.14.0

0.13.1

Bug Fix

• For notebook environments, packages that were accidentally imported but not used were causing a ModuleNotFoundError

0.13.0

Release Notes

This release adds support for collections in Purview and fixes a bug in discovery.search_entities

Some great contributions from @fpvmorais , @vincentk , and@xiaoyongzhu since the last release.

New Features

• Support the core collections APIs:
  • collections.upload_single_entity lets you create a single entity with a specific collection id.
  • collections.upload_entities lets you create multiple entities with a specific collection id.
  • collections.move_entities lets you move a list of guids to a specific collection based on collection id.
  • collections.list_collections lets you list all of the collections so you can find the friendly name and id.

Please note that collection id is required in the upload and move methods which is NOT the friendly name. Use the list_collections method to get the id of your collection.

Bug Fixes

• discovery.search_entities was calling the search api an extra time if the results were already small

Breaking Changes

• None

0.12.0

Release Notes

This release completes support for Apache Atlas 2.2 supported features in Azure Purview.

New Features

Business Metadata

• Deleting a specific business attribute from an entity

resp = client.delete_businessMetadata(
  guid="f222242b-e304-4123-9747-47f6f6f60000",
  businessMetadata={"operations":{"expenseCode":""}}
  )
print(resp)

• Add Business Metadata to an Existing Entity

resp = client.update_businessMetadata(
  guid="f222242b-e304-4123-9747-47f6f6f60000",
  businessMetadata={
    "operations":{"expenseCode":"1011"}
  }
)

print(resp)

Highlighting Existing Features for Atlas 2.2 Support

Business Metadata

• Creating a BusinessMetadata typedef

from pyapacheatlas.core.typedef import AtlasAttributeDef, AtlasStructDef, TypeCategory
bizdef = AtlasStructDef(
    name="operations",
    category=TypeCategory.BUSINESSMETADATA,
    attributeDefs=[
        AtlasAttributeDef(name="expenseCode",options={"maxStrLength": "500","applicableEntityTypes":"[\"DataSet\"]"}),
        AtlasAttributeDef(name="criticality",options={"maxStrLength": "500", "applicableEntityTypes":"[\"DataSet\"]"})
    ]
)
resp = client.upload_typedefs(businessMetadataDefs=[bizdef])
print(resp)

• Add Business Metadata to a New Entity

entity = AtlasEntity(
  name="mytable",
  typeName="azure_sql_table",
  qualified_name="mssql://myserver/mydb/myschema/mytable",
  guid="-1"
)
entity.addBusinessAttribute(operations={"expenseCode":"123", "criticality":"low"})
resp = client.upload_entities([entity])
print(resp)

Custom Attributes

• Add a Custom Attribute to a New Entity

entity = AtlasEntity(
  name="mycustomtable",
  typeName="azure_sql_table",
  qualified_name="mssql://myserver/mydb/myschema/mycustomtable",
  guid="-1"
)
entity.addCustomAttribute(foo="bar", buz="qux")
resp = client.upload_entities([entity])
print(resp)

• Add a Custom Attribute to an Existing Entity
  • This really stinks because you are using the /entity endpoint and must specify all required attributes.
  • You may consider using a client.get_single_entity for a given entity and extract the existing custom attributes.

  existing_entity = AtlasEntity(
    name="mycustomtable",
    typeName="azure_sql_table",
    qualified_name="mssql://myserver/mydb/myschema/mycustomtable",
    guid="-1"
  )
  # Note that we've changed foo to baz which will update foo and we still specify buz to keep it around
  existing_entity.addCustomAttribute(foo="baz", buz="qux")
  resp = client.upload_entities([existing_entity])
  print(resp)

• Delete a Custom Attribute from an Existing Entity
  • See comments above, you're just updating the entity without the or any of the custom attributes.

  existing_entity = AtlasEntity(
    name="mycustomtable",
    typeName="azure_sql_table",
    qualified_name="mssql://myserver/mydb/myschema/mycustomtable",
    guid="-1"
  )
  # Note we've omitted the foo attribute meaning we will delete it
  existing_entity.addCustomAttribute(buz="qux")
  resp = client.upload_entities([existing_entity])
  print(resp)

Custom Labels

• Add Custom Labels to a New Entity

entity = AtlasEntity(
  name="mylabeledtable",
  typeName="azure_sql_table",
  qualified_name="mssql://myserver/mydb/myschema/mylabeledtable",
  guid="-1",
  labels= ["a", "b", "c"]
)
resp = client.upload_entities([entity])
print(resp)

• Append new custom labels (without overwriting)

resp = client.update_entity_labels(labels=['d','e'], 
  guid="57b23112-d665-44c5-afb0-b4f6f6f60000", 
  force_update=False
)
print(resp)

• Completely overwrite custom labels with a new set of labels

resp = client.update_entity_labels(labels=['j','k', 'l'], 
  guid="57b23112-d665-44c5-afb0-b4f6f6f60000", 
  force_update=True
)
print(resp)

• Remove one or many Custom Labels

resp = client.delete_entity_labels(labels=['j','k'], 
  guid="57b23112-d665-44c5-afb0-b4f6f6f60000", 
  force_update=True
)
print(resp)

Bug Fixes

• An error was occuring when using upload_typedefs and the force_update=True flag was set.

Breaking Changes

• An internal method _get_typedefs_header modified its response for busienss_metadataDefs to businessMetadataDefs.

0.11.0

New Features

• Enable passing arguments to the underlying requests package. This supports #181 by enabling a configuration such as:

client = PurviewClient(
  purview_name="myservicename",
  authentication=cred,
  requests_verify=False
)

• The search_entities method now accepts a body parameter which acts like the body in the query method. (#184)

Breaking Changes

• Deprecated TablesLineage and FineGrainColumnLineage from the Excel template. To reenable these tabs, pass --include-deprecated to the pyapacheatlas -m --make-template path.xlsx command or include_deprecated=True to the make_template() method (#188)

Bug Fixes

• The Excel sheet was failing when only owners was provided and experts was omitted. The owners column can now be specified by itself and the ExcelReader can properly parse the column. (#183)
• When defining an AtlasAttributeDef you can now pass in any of the Cardinality enum options and the AtlasAttributeDef will extract the enum value automatically (#185).

0.10.0

New Features

• Added support for Purview's DiscoveryREST API endpoints

Breaking Changes

• PurviewClient.search_entities is now deprecated. Please migrate to PurviewClient.discover.search_entities.

0.9.1

Bug Fixes

• GlossaryClient.assignTerm incorrectly referenced get_glossary_term when using termName.
• GlossaryClient.delete_assignedTerm incorrectly referenced get_glossary_term when using termName.
• GlossaryClient.get_termAssignedEntities incorrectly referenced get_glossary_term when using termName.

Minor Changes

• GlossaryClient.assignTerm now supports passing in a glossary_guid to speed up operations when using termName.
• GlossaryClient.delete_assignedTerm now supports passing in a glossary_guid to speed up operations when using termName.
• GlossaryClient.get_termAssignedEntities now supports passing in a glossary_guid to speed up operations when using termName.

0.9.0

New Features

MS Graph and Helping to Support Expert and Owner Lookup

• For Purview users, introduced a utility function for the MS Graph, making it easier to look up user object ids.
  • PurviewClient.msgraph.upn_to_id
  • PurviewClient.msgraph.email_to_id
• After introducing these utility functions, the ExcelReader.parse_bulk_entities method takes a contacts_func.
  • The contacts_func parameter takes a function and will be executed for every contact provided in the experts and owners columns.

AtlasObjectId to Avoid Updating Existing Required Relationships

• The ExcelReader.parse_bulk_entities now supports using an AtlasObjectId as a reference in the [Relationship].

  • For example, if you wanted to reference an existing entity without uploading it in the BulkEntities tab, you would use...
  • AtlasObjectId(guid:xx-yy-zz) or AtlasObjectId(typeName:someType qualifiedName:someQualifiedName).
  • This would be the value inside of a [Relationship] xyz column where xyz is the relationship attribute name.
  • This is really useful when you have a required relationship (like azure_sql_schema for an azure_sql_table) that also has a required relationship.
  • This lets you reference only the first dependency and you don't risk changing anything like the name or having to deal with required types.

• The PurviewClient and AtlasClient now have a glossary property which supports many of the glossary endpoints.

• In addition a PurviewGlossaryTerm and an AtlasGlossaryTerm class are provided.

  • PurviewGlossaryTerm supports term hierarchy with the add_hierarchy method.

• For current Atlas users, added support for defining businessMetadataDefs.

• For current Atlas users, added support for the label endpoints by introducing:

  • AtlasClient.update_entity_labels
  • AtlasClient.delete_entity_labels

• For current Atlas users (and one day Purview users too), added support for customAttributes in the excel upload by using a column header [custom] myAttribute.

Breaking Changes

• Added a deprecation warning around the AtlasClient glossary methods. Users should migrate to AtlasClient.glossary or PurviewClient.glossary.