Skip to content

Instantly share code, notes, and snippets.

View djtfmartin's full-sized avatar

Dave Martin djtfmartin

25 followers · 13 following
View GitHub Profile
@djtfmartin
djtfmartin / sample-occ.tsv
Last active July 28, 2025 10:52
We can make this file beautiful and searchable if this error is corrected: It looks like row 4 should actually have 20 columns, instead of 13 in line 3.
occurrenceID eventDate decimalLatitude decimalLongitude materialSampleID recordedBy kingdom phylum class order family genus scientificName taxonID sampleSizeValue sampleSizeUnit organismQuantity organismQuantityType basisOfRecord eventID
102.100.100/10601:00d2d306d387c11e63eac2b97f184ac8 2013-04-09 -10.495855 105.566286 102.100.100/10601 Fungi Ascomycota Dothideomycetes Pleosporales Pleosporaceae Alternaria Alternaria ASV:00d2d306d387c11e63eac2b97f184ac8 1957 DNA sequence reads 1 DNA sequence reads MATERIAL_SAMPLE 102.100.100/10601
102.100.100/10601:02e8e53ad96c9864c41b2cf6fd0aa169 2013-04-09 -10.495855 105.566286 102.100.100/10601 Fungi Ascomycota Dothideomycetes Pleosporales Corynesporascaceae Corynespora Corynespora cassiicola ASV:02e8e53ad96c9864c41b2cf6fd0aa169 1957 DNA sequence reads 2 DNA sequence reads MATERIAL_SAMPLE 102.100.100/10601
102.100.100/10601:030608bc1fb64907e5e37a7913cd7ec0 2013-04-09 -10.495855 105.566286 102.100.100/10601 Plantae Tracheophyta Magnoliopsida Apiales Araliaceae Panax Pan
@djtfmartin
djtfmartin / sample-occ.csv
Created July 28, 2025 10:47
sample-occ.csv
We can make this file beautiful and searchable if this error is corrected: No commas found in this CSV file in line 0.
occurrenceID eventDate decimalLatitude decimalLongitude materialSampleID recordedBy kingdom phylum class order family genus scientificName taxonID sampleSizeValue sampleSizeUnit organismQuantity organismQuantityType basisOfRecord eventID
102.100.100/10601:00d2d306d387c11e63eac2b97f184ac8 2013-04-09 -10.495855 105.566286 102.100.100/10601 Fungi Ascomycota Dothideomycetes Pleosporales Pleosporaceae Alternaria Alternaria ASV:00d2d306d387c11e63eac2b97f184ac8 1957 DNA sequence reads 1 DNA sequence reads MATERIAL_SAMPLE 102.100.100/10601
102.100.100/10601:02e8e53ad96c9864c41b2cf6fd0aa169 2013-04-09 -10.495855 105.566286 102.100.100/10601 Fungi Ascomycota Dothideomycetes Pleosporales Corynesporascaceae Corynespora Corynespora cassiicola ASV:02e8e53ad96c9864c41b2cf6fd0aa169 1957 DNA sequence reads 2 DNA sequence reads MATERIAL_SAMPLE 102.100.100/10601
102.100.100/10601:030608bc1fb64907e5e37a7913cd7ec0 2013-04-09 -10.495855 105.566286 102.100.100/10601 Plantae Tracheophyta Magnoliopsida Apiales Araliaceae Panax Pan
@djtfmartin
djtfmartin / weblog_v1_species.sql
Last active August 19, 2024 13:41
weblog v1 species usage
SELECT
SUM(CASE WHEN requestpath LIKE '/v1/species/match%' THEN 1 ELSE 0 END) AS match,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/vernacularNames' THEN 1 ELSE 0 END) AS vernacularNames,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/verbatim' THEN 1 ELSE 0 END) AS verbatim,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/typeSpecimens' THEN 1 ELSE 0 END) AS typeSpecimens,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/toc' THEN 1 ELSE 0 END) AS toc_count,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/synonyms' THEN 1 ELSE 0 END) AS synonyms_count,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/speciesProfiles' THEN 1 ELSE 0 END) AS speciesProfiles,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/related' THEN 1 ELSE 0 END) AS related,
SUM(CASE WHEN requestpath LIKE '/v1/species/%/references' THEN 1 ELSE 0 END) AS species_references,
@djtfmartin
djtfmartin / gist:551e632d75679b574bdc037abaa79208
Created June 29, 2023 15:52
species list clean up
update species_list set is_private=true WHERE list_name REGEXP '^[0-9]' and list_type='OTHER' and is_authoritative is null;
update species_list set is_private=true WHERE list_name REGEXP '^[0-9]' and list_type='SPATIAL_PORTAL' and is_authoritative is null;
update species_list set is_private=true WHERE list_name like 'My species list%' and list_type='OTHER' and is_authoritative is null;
update species_list set is_private=true WHERE list_name = 'My species list' and list_type='SPATIAL_PORTAL' and is_authoritative is null;
update species_list set is_private=true WHERE list_name like 'My species list%' and list_type='SPATIAL_PORTAL' and is_authoritative is null;
update species_list set is_private=true WHERE list_name = 'My species list' and is_authoritative is null;
update species_list set is_private=true
WHERE
( list_name like 'My species list%'
OR list_name = 'My species list'
@djtfmartin
djtfmartin / export_of_raw_db_fields_2018-06-13.txt
Created June 13, 2018 22:29
export_of_raw_db_fields_2018-06-13.txt
_class
abcdIdentificationQualifier
abcdIdentificationQualifierInsertionPoint
abcdTypeStatus
acceptedNameUsage
associatedMedia
associatedOccurrences
associatedReferences
associatedSequences
associatedTaxa
@djtfmartin
djtfmartin / export_of_db_fields_2018-06-13.txt
Created June 13, 2018 22:26
export_of_db_fields_2018-06-13.txt
_class
_qa
abcdIdentificationQualifier
abcdIdentificationQualifier_p
abcdIdentificationQualifierInsertionPoint
abcdTypeStatus
acceptedNameUsage
associatedMedia
associatedOccurrences
associatedOccurrences_p
@djtfmartin
djtfmartin / gist:5064a77eeb7533e2d23d26c446ca51a0
Created November 16, 2017 11:45
Index sync script
import groovy.json.JsonSlurper
statusUrl = new URL("http://XXXXXXXX:8983/solr/admin/collections?action=clusterstatus&wt=json")
slurper = new JsonSlurper()
json = slurper.parseText(statusUrl.text)
collectionName = "biocache"
println("Parsing")
@djtfmartin
djtfmartin / gist:3a98a0f4339adea3444cf1ff761b18a1
Created November 16, 2017 11:31
FixDatePrecision.scala
package au.org.ala.biocache.tool
import au.org.ala.biocache.Config
import org.slf4j.LoggerFactory
object FixDatePrecision extends au.org.ala.biocache.cmd.Tool {
def cmd = "fix-date-precision"
def desc = "Cleanup old date precision values."
@djtfmartin
djtfmartin / gist:4a639e601ba4f156a8953c69cfbafef7
Created September 19, 2017 19:22
2017-09-20 02:07:07,477 INFO [GbifRegistryService] Updating GBIF organisation dp46: 8dda5e56-c4b4-4af2-90f9-170d9d22f2c1
2017-09-20 02:07:08,179 INFO [GbifRegistryService] Setting GBIF country of attribution to AU
2017-09-20 02:07:09,030 INFO [GbifRegistryService] Successfully updated provider in GBIF: 8dda5e56-c4b4-4af2-90f9-170d9d22f2c1
2017-09-20 02:07:09,839 INFO [GbifRegistryService] Removing contacts
2017-09-20 02:07:10,670 INFO [GbifRegistryService] Removed contact 615477
2017-09-20 02:07:10,671 INFO [GbifRegistryService] Successfully synced contacts: 8dda5e56-c4b4-4af2-90f9-170d9d22f2c1
2017-09-20 02:07:10,674 INFO [GbifRegistryService] Creating GBIF resource for dr797
2017-09-20 02:07:10,674 INFO [GbifRegistryService] Creating dataset in GBIF: null
2017-09-20 02:07:10,674 WARN [GbifRegistryService] Unable to register dataset - please check license: dr797 : Whicher Scarp Flora Survey : other
2017-09-20 02:07:10,674 INFO [GbifRegistryService] Registry key not set for resource: dr797. Not sy
@djtfmartin
djtfmartin / loading_nvc_table.sql
Created August 1, 2017 10:36
create table nvc_scotland (pid integer, name varchar(255), the_geom geometry, the_geom_4326 geometry);
insert into nvc_scotland(pid, name, the_geom)
SELECT nextval('objects_id_seq'::regclass), commun_cat, ST_UNION(the_geom) FROM "65" where commun_cat='A, Aquatic Communities' GROUP BY commun_cat;
insert into nvc_scotland(pid, name, the_geom)
SELECT nextval('objects_id_seq'::regclass), commun_cat, ST_UNION(the_geom) FROM "65" where commun_cat='W, Woodlands' GROUP BY commun_cat;
# Doesnt return.....
insert into nvc_scotland(pid, name, the_geom)