CellMarker

lamindb provides access to the following public cell marker ontologies through bionty:

  1. CellMarker

Here we show how to access and search cell marker ontologies to standardize new data.

import bionty as bt
import pandas as pd

PublicOntology objects

Let us create a public ontology accessor with public(), which chooses a default public ontology source from Source. It’s a PublicOntology object, which you can think about as a public registry:

public = bt.CellMarker.public(organism="human")
public

→ connected lamindb: testuser1/test-public-ontologies

PublicOntology
Entity: CellMarker
Organism: human
Source: cellmarker, 2.0
#terms: 15466

As for registries, you can export the ontology as a DataFrame:

df = public.df()
df.head()
name synonyms gene_symbol ncbi_gene_id uniprotkb_id
0 A1BG A1BG 1 P04217
1 A2M A2M 3494 None
2 A2ML1 A2ML1 144568 A8K2U0
3 A4GALT A4GALT 53947 A0A0S2Z5J1
4 AADAC AADAC 13 P22760

Unlike registries, you can also export it as a Pronto object via public.ontology.

Look up terms

As for registries, terms can be looked up with auto-complete:

lookup = public.lookup()

The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):

lookup.immp1l

CellMarker(name='IMMP1L', synonyms='', gene_symbol='IMMP1L', ncbi_gene_id='196294', uniprotkb_id='Q96LU5')

To look up the exact original strings, convert the lookup object to dict and use the [] accessor:

lookup_dict = lookup.dict()
lookup_dict["IMMP1L"]

CellMarker(name='IMMP1L', synonyms='', gene_symbol='IMMP1L', ncbi_gene_id='196294', uniprotkb_id='Q96LU5')

Search terms

Search behaves in the same way as it does for registries:

public.search("CD4").head(5)
synonyms gene_symbol ncbi_gene_id uniprotkb_id __ratio__
name
Cd4 CD4 920 B4DT49 100.0
CD4+ None None None 100.0
CD45RB None None None 90.0
CD45RO None None None 90.0
CD44R None None None 90.0

Search another field (default is .name):

public.search("CD4", field=public.gene_symbol).head(1)
name synonyms ncbi_gene_id uniprotkb_id __ratio__
gene_symbol
CD4 Cd4 920 B4DT49 100.0

Standardize cell marker identifiers

Let us generate a DataFrame that stores a number of cell markers identifiers, some of which corrupted:

markers = pd.DataFrame(
    index=[
        "KI67",
        "CCR7",
        "CD14",
        "CD8",
        "CD45RA",
        "CD4",
        "CD3",
        "CD127a",
        "PD1",
        "Invalid-1",
        "Invalid-2",
        "CD66b",
        "Siglec8",
        "Time",
    ]
)

Now let’s check which cell markers can be found in the reference:

public.inspect(markers.index, public.name);

! 8 terms (57.10%) are not validated for name: KI67, CCR7, CD14, CD4, CD127a, Invalid-1, Invalid-2, Time

   detected 4 terms with inconsistent casing/synonyms: KI67, CCR7, CD14, CD4

→  standardize terms via .standardize()

Logging suggests to map synonyms:

synonyms_mapper = public.standardize(markers.index, return_mapper=True)
synonyms_mapper

{'KI67': 'Ki67', 'CCR7': 'Ccr7', 'CD14': 'Cd14', 'CD4': 'Cd4'}

Let’s replace the synonyms with standardized names in the DataFrame:

markers.rename(index=synonyms_mapper, inplace=True)

The Time, Invalid-1 and Invalid-2 are non-marker channels which won’t be curated by cell marker:

public.inspect(markers.index, public.name);

! 4 terms (28.60%) are not validated for name: CD127a, Invalid-1, Invalid-2, Time

We don’t find CD127a, let’s check in the lookup with auto-completion:

lookup = public.lookup()
lookup.cd127

CellMarker(name='CD127', synonyms='', gene_symbol='IL7R', ncbi_gene_id='3575', uniprotkb_id='P16871', _5='cd127')

It should be cd127, we had a typo there with cd127a:

curated_df = markers.rename(index={"CD127a": lookup.cd127.name})

Optionally, search:

public.search("CD127a").head()
synonyms gene_symbol ncbi_gene_id uniprotkb_id __agg__ __ratio__
name
CD127 IL7R 3575 P16871 cd127 90.909091
CD1 CD1A 910 P29016 cd1 90.000000
CD172a None None None cd172a 83.333333
CD167a None None None cd167a 83.333333
CD121a None None None cd121a 83.333333

Now we see that all cell marker candidates validate:

public.validate(curated_df.index, public.name);

! 3 terms (21.40%) are not validated: Invalid-1, Invalid-2, Time

Ontology source versions

For any given entity, we can choose from a number of versions:

bt.CellMarker.list_source()
Hide code cell output 
<QuerySet [Source(uid='3kDh', entity='bionty.CellMarker', organism='human', name='cellmarker', version='2.0', in_db=False, currently_used=True, description='CellMarker', url='s3://bionty-assets/human_cellmarker_2.0_CellMarker_lookup.parquet', md5='d565d4a542a5c7e7a06255975358e4f4', source_website='http://bio-bigdata.hrbmu.edu.cn/CellMarker', created_by_id=1, updated_at='2024-08-28 16:06:12 UTC'), Source(uid='7bV5', entity='bionty.CellMarker', organism='mouse', name='cellmarker', version='2.0', in_db=False, currently_used=True, description='CellMarker', url='s3://bionty-assets/mouse_cellmarker_2.0_CellMarker_lookup.parquet', md5='189586732c63be949e40dfa6a3636105', source_website='http://bio-bigdata.hrbmu.edu.cn/CellMarker', created_by_id=1, updated_at='2024-08-28 16:06:12 UTC')]>

# only lists the sources that are currently used
bt.CellMarker.list_source(currently_used=True).df()
uid entity organism name version in_db currently_used description url md5 source_website dataframe_artifact_id run_id created_by_id updated_at
id
28 3kDh bionty.CellMarker human cellmarker 2.0 False True CellMarker s3://bionty-assets/human_cellmarker_2.0_CellMa... d565d4a542a5c7e7a06255975358e4f4 http://bio-bigdata.hrbmu.edu.cn/CellMarker None None 1 2024-08-28 16:06:12.912582+00:00
29 7bV5 bionty.CellMarker mouse cellmarker 2.0 False True CellMarker s3://bionty-assets/mouse_cellmarker_2.0_CellMa... 189586732c63be949e40dfa6a3636105 http://bio-bigdata.hrbmu.edu.cn/CellMarker None None 1 2024-08-28 16:06:12.912619+00:00

When instantiating a Bionty object, we can choose a source or version:

source = bt.Source.get(
    name="cellmarker", version="2.0", organism="human"
)
public = bt.CellMarker.public(source=source)
public

PublicOntology
Entity: CellMarker
Organism: human
Source: cellmarker, 2.0
#terms: 15466

The currently used ontologies can be displayed using:

bt.Source.filter(currently_used=True).df()
Hide code cell output
uid entity organism name version in_db currently_used description url md5 source_website dataframe_artifact_id run_id created_by_id updated_at
id
1 33TU bionty.Organism vertebrates ensembl release-112 False True Ensembl https://ftp.ensembl.org/pub/release-112/specie... 0ec37e77f4bc2d0b0b47c6c62b9f122d https://www.ensembl.org None None 1 2024-08-28 16:06:12.911561+00:00
6 6bbV bionty.Organism bacteria ensembl release-57 False True Ensembl https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... ee28510ed5586ea7ab4495717c96efc8 https://www.ensembl.org None None 1 2024-08-28 16:06:12.911772+00:00
7 6s9n bionty.Organism fungi ensembl release-57 False True Ensembl http://ftp.ensemblgenomes.org/pub/fungi/releas... dbcde58f4396ab8b2480f7fe9f83df8a https://www.ensembl.org None None 1 2024-08-28 16:06:12.911810+00:00
8 2PmT bionty.Organism metazoa ensembl release-57 False True Ensembl http://ftp.ensemblgenomes.org/pub/metazoa/rele... 424636a574fec078a61cbdddb05f9132 https://www.ensembl.org None None 1 2024-08-28 16:06:12.911848+00:00
9 7GPH bionty.Organism plants ensembl release-57 False True Ensembl https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... eadaa1f3e527e4c3940c90c7fa5c8bf4 https://www.ensembl.org None None 1 2024-08-28 16:06:12.911886+00:00
10 4tsk bionty.Organism all ncbitaxon 2023-06-20 False True NCBItaxon Ontology s3://bionty-assets/df_all__ncbitaxon__2023-06-... 00d97ba65627f1cd65636d2df22ea76c https://github.com/obophenotype/ncbitaxon None None 1 2024-08-28 16:06:12.911923+00:00
11 4UGN bionty.Gene human ensembl release-112 False True Ensembl s3://bionty-assets/df_human__ensembl__release-... 4ccda4d88720a326737376c534e8446b https://www.ensembl.org None None 1 2024-08-28 16:06:12.911961+00:00
15 4r4f bionty.Gene mouse ensembl release-112 False True Ensembl s3://bionty-assets/df_mouse__ensembl__release-... 519cf7b8acc3c948274f66f3155a3210 https://www.ensembl.org None None 1 2024-08-28 16:06:12.912109+00:00
19 4RPA bionty.Gene saccharomyces cerevisiae ensembl release-112 False True Ensembl s3://bionty-assets/df_saccharomyces cerevisiae... 11775126b101233525a0a9e2dd64edae https://www.ensembl.org None None 1 2024-08-28 16:06:12.912255+00:00
22 3EYy bionty.Protein human uniprot 2024-03 False True Uniprot s3://bionty-assets/df_human__uniprot__2024-03_... b5b9e7645065b4b3187114f07e3f402f https://www.uniprot.org None None 1 2024-08-28 16:06:12.912364+00:00
25 01RW bionty.Protein mouse uniprot 2024-03 False True Uniprot s3://bionty-assets/df_mouse__uniprot__2024-03_... b1b6a196eb853088d36198d8e3749ec4 https://www.uniprot.org None None 1 2024-08-28 16:06:12.912473+00:00
28 3kDh bionty.CellMarker human cellmarker 2.0 False True CellMarker s3://bionty-assets/human_cellmarker_2.0_CellMa... d565d4a542a5c7e7a06255975358e4f4 http://bio-bigdata.hrbmu.edu.cn/CellMarker None None 1 2024-08-28 16:06:12.912582+00:00
29 7bV5 bionty.CellMarker mouse cellmarker 2.0 False True CellMarker s3://bionty-assets/mouse_cellmarker_2.0_CellMa... 189586732c63be949e40dfa6a3636105 http://bio-bigdata.hrbmu.edu.cn/CellMarker None None 1 2024-08-28 16:06:12.912619+00:00
30 6LyR bionty.CellLine all clo 2022-03-21 False True Cell Line Ontology https://data.bioontology.org/ontologies/CLO/su... ea58a1010b7e745702a8397a526b3a33 https://bioportal.bioontology.org/ontologies/CLO None None 1 2024-08-28 16:06:12.912655+00:00
32 1Lhf bionty.CellType all cl 2024-05-15 False True Cell Ontology http://purl.obolibrary.org/obo/cl/releases/202... 8a8638a9e79567935793e5007704c650 https://obophenotype.github.io/cell-ontology None None 1 2024-08-28 16:06:12.912727+00:00
39 MUtA bionty.Tissue all uberon 2024-08-07 False True Uberon multi-species anatomy ontology http://purl.obolibrary.org/obo/uberon/releases... http://obophenotype.github.io/uberon None None 1 2024-08-28 16:06:12.912977+00:00
47 2L2r bionty.Disease all mondo 2024-06-04 False True Mondo Disease Ontology http://purl.obolibrary.org/obo/mondo/releases/... c47e8edb894c01f2511dfe0751fbc428 https://mondo.monarchinitiative.org None None 1 2024-08-28 16:06:12.913268+00:00
54 4ksw bionty.Disease human doid 2024-05-29 False True Human Disease Ontology http://purl.obolibrary.org/obo/doid/releases/2... bbefd72247d638edfcd31ec699947407 https://disease-ontology.org None None 1 2024-08-28 16:06:12.913523+00:00
62 69Xc bionty.ExperimentalFactor all efo 3.66.0 False True The Experimental Factor Ontology http://www.ebi.ac.uk/efo/releases/v3.66.0/efo.owl 6bd24217c740af7e1e771c1dabc9680b https://bioportal.bioontology.org/ontologies/EFO None None 1 2024-08-28 16:06:12.913823+00:00
67 48fB bionty.Phenotype human hp 2024-04-26 False True Human Phenotype Ontology https://github.com/obophenotype/human-phenotyp... e0f2e534eb2ad44a4d45573ef27b508f https://hpo.jax.org None None 1 2024-08-28 16:06:12.916315+00:00
72 4t7Q bionty.Phenotype mammalian mp 2024-06-18 False True Mammalian Phenotype Ontology https://github.com/mgijax/mammalian-phenotype-... 795d8378fe48ec13b41d01a86dd1c86c https://github.com/mgijax/mammalian-phenotype-... None None 1 2024-08-28 16:06:12.916497+00:00
75 sqPX bionty.Phenotype zebrafish zp 2024-04-18 False True Zebrafish Phenotype Ontology https://github.com/obophenotype/zebrafish-phen... 2231ebaa95becf8ff34a33c95a8d4350 https://github.com/obophenotype/zebrafish-phen... None None 1 2024-08-28 16:06:12.916602+00:00
79 6S4q bionty.Phenotype all pato 2024-03-28 False True Phenotype And Trait Ontology http://purl.obolibrary.org/obo/pato/releases/2... 6b1eaacd3d453b34375ce2e31c16328a https://github.com/pato-ontology/pato None None 1 2024-08-28 16:06:12.916748+00:00
81 7Ent bionty.Pathway all go 2024-06-17 False True Gene Ontology https://data.bioontology.org/ontologies/GO/sub... 7fa7ade5e3e26eab3959a7e4bc89ad4f http://geneontology.org None None 1 2024-08-28 16:06:12.916819+00:00
86 3rm9 BFXPipeline all lamin 1.0.0 False True Bioinformatics Pipeline s3://bionty-assets/df_all__lamin__1.0.0__BFXpi... https://lamin.ai None None 1 2024-08-28 16:06:12.916994+00:00
87 ugaI Drug all dron 2024-08-05 False True Drug Ontology https://data.bioontology.org/ontologies/DRON/s... https://bioportal.bioontology.org/ontologies/DRON None None 1 2024-08-28 16:06:12.917028+00:00
91 1GbF bionty.DevelopmentalStage human hsapdv 2024-05-28 False True Human Developmental Stages https://github.com/obophenotype/developmental-... https://github.com/obophenotype/developmental-... None None 1 2024-08-28 16:06:12.917166+00:00
93 10va bionty.DevelopmentalStage mouse mmusdv 2024-05-28 False True Mouse Developmental Stages https://github.com/obophenotype/developmental-... https://github.com/obophenotype/developmental-... None None 1 2024-08-28 16:06:12.917234+00:00
95 MJRq bionty.Ethnicity human hancestro 3.0 False True Human Ancestry Ontology https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro None None 1 2024-08-28 16:06:12.917304+00:00
96 5JnV BioSample all ncbi 2023-09 False True NCBI BioSample attributes s3://bionty-assets/df_all__ncbi__2023-09__BioS... 918db9bd1734b97c596c67d9654a4126 https://www.ncbi.nlm.nih.gov/biosample/docs/at... None None 1 2024-08-28 16:06:12.917339+00:00