Applescript for Archive analysis


#1

Hello, all. Wondering if any of you Applescript wizards can give me a few pointers here. I’ve managed to cobble together a script that allows me to do a quick review of what I’ve been working on. I copy the archive from my main task file, and invoke the script via Textexpander to paste the archive into an NValt note, with totals for each tag. Since the “archive as done” script I’m using automatically grabs the project as a tag, that means I can quickly get a rough idea of how much work I’ve put in on various different areas/projects etc.

And no, I never did get my head around the CLI…

I have two questions. One specific query, and one more nebulous.
1: My command of regular expressions is pretty weak at the moment. The script doesn’t handle multiword tags particularly well, say, for example if the tag has a value e.g. @project(attack the DeathStar:). Any suggestions for an update that might capture tags like this?

TLDR— the reg exp is in this line: “set taglist to find text " @[a-zA-Z]+[(]?[a-zA-Z0-9-]+” in theClipboard with regexp, all occurrences and string result"

2: Any suggestions for more efficient functioning code would be appreciated. Still learning, here.

Here’s the script. Thanks in advance for any suggestions.

-- this counts tasks and projects from the clipboard, but we can switch to front most document for ease... 

set theClipboard to the clipboard as text
set theClipboard to stripChar(theClipboard, tab)

set tid to AppleScript's text item delimiters
set AppleScript's text item delimiters to "
- "
set aList to the text items in the theClipboard
set theTasks to (the count aList) - 1
set AppleScript's text item delimiters to "# "
set aList to the text items in the theClipboard
set theProjects to (the count aList) - 1
set AppleScript's text item delimiters to tid

set taglist to find text " @[a-zA-Z]+[(]?[a-zA-Z0-9-]+" in theClipboard with regexp, all occurrences and string result
set olddelimiter to AppleScript's text item delimiters
set AppleScript's text item delimiters to " "
set AppleScript's text item delimiters to olddelimiter
set nTags to (the count taglist) - 1

set theList to {}
repeat with theWord in (every text item in taglist)
	copy theWord & "—, " & countMatches(taglist, (contents of theWord)) as string to end of theList
end repeat

set tagOccur to removeduplicates(theList)
considering numeric strings
	set tagOccur to simple_sort(tagOccur)
end considering
set tagOccur to replaceString(tagOccur, " @", "
")

set finalStats to "Tasks: " & theTasks & "; Projects: " & theProjects & "
" & tagOccur & " 
---- " & "
" & theClipboard

set the clipboard to finalStats
tell application "System Events" to keystroke "v" using command down

-- FUNCTIONS AND SUBROUTINES

-- account for any tabs

on stripChar(str, chrs)
	tell AppleScript
		set oldTIDs to text item delimiters
		set text item delimiters to characters of chrs
		set TIs to text items of str
		set text item delimiters to ""
		set str to TIs as string
		set text item delimiters to oldTIDs
	end tell
	return str
end stripChar

on countMatches(lst, val)
	local lst, val, counter
	try
		if lst's class is not list then error "not a list." number -1704
		script k
			property l : lst
		end script
		set len to count k's l
		set counter to 0
		repeat with i from 1 to len
			if k's l's item i is val then set counter to counter + 1
		end repeat
		return counter
	on error eMsg number eNum
		error "Can't countMatches: " & eMsg number eNum
	end try
end countMatches

on removeduplicates(lst)
	local lst, itemRef, res, itm
	try
		if lst's class is not list then error "not a list." number -1704
		script k
			property l : lst
			property res : {}
		end script
		repeat with itemRef in k's l
			set itm to itemRef's contents
			-- note: minor speed optimisation when removing duplicates 
			-- from ordered lists: assemble new list in reverse so 
			-- 'contains' operator checks most recent item first
			if k's res does not contain {itm} then ¬
				set k's res's beginning to itm
		end repeat
		return k's res's reverse
	on error eMsg number eNum
		error "Can't removeDuplicates: " & eMsg number eNum
	end try
end removeduplicates

on simple_sort(my_list)
	set the index_list to {}
	set the sorted_list to {}
	repeat (the number of items in my_list) times
		set the low_item to ""
		repeat with i from 1 to (number of items in my_list)
			if i is not in the index_list then
				set this_item to item i of my_list as text
				if the low_item is "" then
					set the low_item to this_item
					set the low_item_index to i
				else if this_item comes before the low_item then
					set the low_item to this_item
					set the low_item_index to i
				end if
			end if
		end repeat
		set the end of sorted_list to the low_item
		set the end of the index_list to the low_item_index
	end repeat
	return the sorted_list
end simple_sort

on replaceString(theText, oldString, newString)
	local ASTID, theText, oldString, newString, lst
	set ASTID to AppleScript's text item delimiters
	try
		considering case
			set AppleScript's text item delimiters to oldString
			set lst to every text item of theText
			set AppleScript's text item delimiters to newString
			set theText to lst as string
		end considering
		set AppleScript's text item delimiters to ASTID
		return theText
	on error eMsg number eNum
		set AppleScript's text item delimiters to ASTID
		error "Can't replaceString: " & eMsg number eNum
	end try
end replaceString

#2

Can you give a quick example archive and then the results that you would expect this script to generate from that archive?


#3

Here’s maybe an alternative approach:

tell front document of application "FoldingText"
  evaluate script "
  function(editor, options) {
    var archived =  editor.tree().evaluateNodePath('//@line:text = Archive//*');
    var archivedCount = archived.length;
    var results = {};
    
    for (var i = 0; i < archivedCount; i++) {
      var each = archived[i];
      var project = each.tag('project') || 'No Project';
      var count = results[project] || 0;
      results[project] = count + 1;
    }
    
    return results;
  }"
end tell

Unless I’m misunderstanding the requirements it seems like your script is doing more work then it really needs to. If you’re goal is to count how many items are in the archive by project then it’s much easier to work directly with FoldingText’s data model then it is to try parsing the text yourself.

This script:

  1. Gets a list of all nodes in the Archive
  2. Loop over those nodes and for each node look for project tag.
  3. Maintain a count for how many nodes you see in each project.

Jesse


#4

Thanks for the response, Jesse!

The script currently spits out something like this:

Tasks: 27; Projects: 2
@done(2015-10-28—, 17
@done(2015-10-29—, 7
@done(2015-10-30—, 4
@today—, 10
@domestics—, 4
@admin—, 8
@project(-----jacobsamlarose—, 7
@project(review—, 3
@sysadmin—, 2

The numbers above probably won’t tally— it’s excised from a real sample, just so you get the rough idea. Also, in real use, the list is sorted by tag alphabetically. I’m now finding it more useful to sort the list so I can see the date range of the archive with the spread of tasks per day (@done + date + n), then tags by descending number of tasks per tag.

The fact that the list can give me a count for number of tasks completed per project is a rather nice by-product of the fact that the “archive done tasks” plug-in captures a task’s project as a tag. What I’m really aiming for, however, is a rough/quick sense of where my activity has been directed, based on the tags associated with each task, not just the project a task belongs to (though that’s obviously also important). It’s not going to be 100% accurate— I don’t tag every single task with a context or GTD-esque “area”— but it’s proved interesting thus far, and I hope will serve as a useful nudge in better directions over an extended period of time. I’m also thinking about how I might be able to use these lists as CSV data to feed some data visualisation… something to kill some time playing with over the Xmas break, perhaps… *grins

Working with FT’s data model seems to make more sense— I’m just finding Applescript easier to get my head around (@complexpoint - I know, Rob, you’ve told me before…)

Haven’t been able to get your script to output anything yet. Will play around with it some more later this evening. Probably something to do with the way that I’m currently invoking it (via TextExpander)?


#5

Off to trick or treat now… but yes, run my script from Script Editor to see output… and the output is just a map (forget proper applescript term), nothing formatted.


#6

Figured out why the new script wasn’t working for me. My archive = “archive:”


#7

Took another quick look at the original AppleScript today, and made a few tweaks. This should capture all tags (including a little fix for totals by date for those of us who have more verbose date/time logging for done tasks).

As per original post, this is currently invoked by copying (cutting) the archive out of my main file and calling a TextExpander snippet that pastes the archive in whichever window has focus (for me, a new nvAlt note) with the calculated totals on top. It’s still not perfect, but works well enough for my current purposes.

Oh, and if this sounds in any way useful to anyone else who’d like to play around with it themselves, the script requires Satimage’s Applescript dictionary for support for regular expressions. And I can’t take credit for much, it’s largely cobbled together from bits and pieces I found.

-- this counts tasks and projects from the clipboard, but we can switch to front most document for ease... 

set theClipboard to the clipboard as text
set theClipboard to stripChar(theClipboard, tab)

set tid to AppleScript's text item delimiters
set AppleScript's text item delimiters to "
- "
set aList to the text items in the theClipboard
set theTasks to (the count aList) - 1
set AppleScript's text item delimiters to "# "
set aList to the text items in the theClipboard
set theProjects to (the count aList) - 1
set AppleScript's text item delimiters to tid

set doneList to find text "@done[(]?[a-zA-Z0-9-]+" in theClipboard with regexp, all occurrences and string result
set taglist to doneList & (find text "@[^done][a-zA-Z]+[(]?[a-zA-Z0-9]+[ (-:'a-zA-Z0-9]+" in theClipboard with regexp, all occurrences and string result)
set olddelimiter to AppleScript's text item delimiters
set AppleScript's text item delimiters to " "
set AppleScript's text item delimiters to olddelimiter
set nTags to (the count taglist) - 1

set theList to {}
repeat with theWord in (every text item in taglist)
	copy theWord & "—, " & countMatches(taglist, (contents of theWord)) as string to end of theList
end repeat

set tagOccur to removeduplicates(theList)
considering numeric strings
	set tagOccur to simple_sort(tagOccur)
end considering
set tagOccur to replaceString(tagOccur, " @", "
")

set finalStats to "Tasks: " & theTasks & "; Projects: " & theProjects & "
" & tagOccur & " 
---- " & "
" & theClipboard

set the clipboard to finalStats
tell application "System Events" to keystroke "v" using command down

-- FUNCTIONS AND SUBROUTINES

-- account for any tabs

on stripChar(str, chrs)
	tell AppleScript
		set oldTIDs to text item delimiters
		set text item delimiters to characters of chrs
		set TIs to text items of str
		set text item delimiters to ""
		set str to TIs as string
		set text item delimiters to oldTIDs
	end tell
	return str
end stripChar

on countMatches(lst, val)
	local lst, val, counter
	try
		if lst's class is not list then error "not a list." number -1704
		script k
			property l : lst
		end script
		set len to count k's l
		set counter to 0
		repeat with i from 1 to len
			if k's l's item i is val then set counter to counter + 1
		end repeat
		return counter
	on error eMsg number eNum
		error "Can't countMatches: " & eMsg number eNum
	end try
end countMatches

on removeduplicates(lst)
	local lst, itemRef, res, itm
	try
		if lst's class is not list then error "not a list." number -1704
		script k
			property l : lst
			property res : {}
		end script
		repeat with itemRef in k's l
			set itm to itemRef's contents
			-- note: minor speed optimisation when removing duplicates 
			-- from ordered lists: assemble new list in reverse so 
			-- 'contains' operator checks most recent item first
			if k's res does not contain {itm} then ¬
				set k's res's beginning to itm
		end repeat
		return k's res's reverse
	on error eMsg number eNum
		error "Can't removeDuplicates: " & eMsg number eNum
	end try
end removeduplicates

on simple_sort(my_list)
	set the index_list to {}
	set the sorted_list to {}
	repeat (the number of items in my_list) times
		set the low_item to ""
		repeat with i from 1 to (number of items in my_list)
			if i is not in the index_list then
				set this_item to item i of my_list as text
				if the low_item is "" then
					set the low_item to this_item
					set the low_item_index to i
				else if this_item comes before the low_item then
					set the low_item to this_item
					set the low_item_index to i
				end if
			end if
		end repeat
		set the end of sorted_list to the low_item
		set the end of the index_list to the low_item_index
	end repeat
	return the sorted_list
end simple_sort

on replaceString(theText, oldString, newString)
	local ASTID, theText, oldString, newString, lst
	set ASTID to AppleScript's text item delimiters
	try
		considering case
			set AppleScript's text item delimiters to oldString
			set lst to every text item of theText
			set AppleScript's text item delimiters to newString
			set theText to lst as string
		end considering
		set AppleScript's text item delimiters to ASTID
		return theText
	on error eMsg number eNum
		set AppleScript's text item delimiters to ASTID
		error "Can't replaceString: " & eMsg number eNum
	end try
end replaceString

Next, I’m going to try tweak Jesse’s offered code to see if I can get the same result more efficiently. I can see that, rather than each.tag(‘project’), I need to grab all of the tags for the archive node. Just not exactly sure how to do that, and I can’t simply use a wildcard instead of ‘project’…

Also, at some point, I’d like to make it a more natural part of my archiving workflow— to run a script and have a new text file automatically generated with the captured archive and associated stats.


#8

Grab all of the tags… I think you want to use the tags() method. To find that and more:

  1. FoldingText > Help > Software Development Kit
  2. Click “Documentation” Link
  3. Click “Node” from classes list.
  4. And there you’ll find all the API that you can use for talking to individual nodes.

Jesse


#9

as in: " var myTags = each.tags() || ‘No Tags’;" ?

Tried that in some earlier fumbling around and got no result…


#10

Here’s an example iterating over both the list items in the archive and over all tags in those items:

function summerizeArchive(editor, options) {
	var archivedListItems = editor.tree().evaluateNodePath('//@line:text = Archive//unordered*');
	var tagCounts = {};
	for (var i = 0; i < archivedListItems.length; i++) {
		var item = archivedListItems[i];
		var tagsToValues = item.tags();
		for (var tagName in tagsToValues) {
			var tagValue = tagsToValues[tagName];
			var tagKey = tagName + ': ' + tagValue;
			var count = tagCounts[tagKey] || 0;
			tagCounts[tagKey] = count + 1;
		}
	}
	return tagCounts;	
}

Application('FoldingText').documents[0].evaluate({script: summerizeArchive.toString()});

I changed this example to fully use JavaScript syntax. So using ScriptEditor you’ll need to set it in JavaScript syntax mode.


#11

It occurs to me that I never did post an update on this. I managed to get the Applescript running pretty much as I wanted it, then lost interest in trying to progress further with understanding FT’s data model. Maybe when I’ve got a bit more free time to play with. Nonetheless, here’s the most recent version of this script. I tend to archive monthly. I’ll select everything in the archive, cut, head over to nvALT, write “Archive [YYYY-MM]”, then tap “.sum” nvALT’s text window. That’s a shortcut to invoke the script, which does all the math and spits out the output.

The one remaining flaw I can see with my current regex is that if you have any email addresses in the archived items, it’ll misidentify parts of email addresses as tags. Nothing I can’t ignore.

Hope someone else finds this useful. If anyone else is interested in applying some QS thinking to their list of done items, let me know— I’d be interested in swapping notes.

Finally: while I haven’t (yet) rewritten along the lines of his suggestions, thanks to Jesse for taking the time out to offer some thinking.

-- this counts tasks and projects from the clipboard (though could draw from front most document for ease...)
-- requires Satimage osax to enable regular expressions in Applescript:
--- http://www.satimage.fr/software/en/downloads/downloads_companion_osaxen.html

set theClipboard to the clipboard as text
set theClipboard to stripChar(theClipboard, tab)

set tid to AppleScript's text item delimiters
set AppleScript's text item delimiters to "
- "
set aList to the text items in the theClipboard
set AppleScript's text item delimiters to "# "
set aList to the text items in the theClipboard
set theProjects to (the count aList) - 1
set AppleScript's text item delimiters to tid

set doneRaw to find text "@done[(]?[a-zA-Z0-9-]+" in theClipboard with regexp, all occurrences and string result
set theTasks to (the count doneRaw) - theProjects
set doneList to {}
repeat with theWord in (every text item in doneRaw)
	set end of doneList to theWord & ")"
end repeat

set taglist to doneList & (find text "@([^done][a-zA-Z]+[(].*?[)])|@[^done][a-zA-Z]?[(]?[a-zA-Z0-9]+" in theClipboard with regexp, all occurrences and string result)
set olddelimiter to AppleScript's text item delimiters
set AppleScript's text item delimiters to " "
set AppleScript's text item delimiters to olddelimiter
set nTags to (the count taglist) - 1

set theList to {}
repeat with theWord in (every text item in taglist)
	copy theWord & "  : " & countMatches(taglist, (contents of theWord)) as string to end of theList
end repeat

set tagOccur to removeduplicates(theList)
considering numeric strings
	set tagOccur to simple_sort(tagOccur)
end considering
set tagOccur to replaceString(tagOccur, " @", "
")

if theProjects = 1 then
	set pLabel to "COMPLETED PROJECT: "
else
	set pLabel to "COMPLETED PROJECTS: "
end if

set finalStats to "COMPLETED TASKS: " & theTasks & "; " & pLabel & theProjects & "
" & tagOccur & " 

---- " & "
" & theClipboard

set the clipboard to finalStats
tell application "System Events" to keystroke "v" using command down

-- FUNCTIONS AND SUBROUTINES

-- account for any tabs

on stripChar(str, chrs)
	tell AppleScript
		set oldTIDs to text item delimiters
		set text item delimiters to characters of chrs
		set TIs to text items of str
		set text item delimiters to ""
		set str to TIs as string
		set text item delimiters to oldTIDs
	end tell
	return str
end stripChar

on countMatches(lst, val)
	local lst, val, counter
	try
		if lst's class is not list then error "not a list." number -1704
		script k
			property l : lst
		end script
		set len to count k's l
		set counter to 0
		repeat with i from 1 to len
			if k's l's item i is val then set counter to counter + 1
		end repeat
		return counter
	on error eMsg number eNum
		error "Can't countMatches: " & eMsg number eNum
	end try
end countMatches

on removeduplicates(lst)
	local lst, itemRef, res, itm
	try
		if lst's class is not list then error "not a list." number -1704
		script k
			property l : lst
			property res : {}
		end script
		repeat with itemRef in k's l
			set itm to itemRef's contents
			-- note: minor speed optimisation when removing duplicates 
			-- from ordered lists: assemble new list in reverse so 
			-- 'contains' operator checks most recent item first
			if k's res does not contain {itm} then ¬
				set k's res's beginning to itm
		end repeat
		return k's res's reverse
	on error eMsg number eNum
		error "Can't removeDuplicates: " & eMsg number eNum
	end try
end removeduplicates

on simple_sort(my_list)
	set the index_list to {}
	set the sorted_list to {}
	repeat (the number of items in my_list) times
		set the low_item to ""
		repeat with i from 1 to (number of items in my_list)
			if i is not in the index_list then
				set this_item to item i of my_list as text
				if the low_item is "" then
					set the low_item to this_item
					set the low_item_index to i
				else if this_item comes before the low_item then
					set the low_item to this_item
					set the low_item_index to i
				end if
			end if
		end repeat
		set the end of sorted_list to the low_item
		set the end of the index_list to the low_item_index
	end repeat
	return the sorted_list
end simple_sort

on replaceString(theText, oldString, newString)
	local ASTID, theText, oldString, newString, lst
	set ASTID to AppleScript's text item delimiters
	try
		considering case
			set AppleScript's text item delimiters to oldString
			set lst to every text item of theText
			set AppleScript's text item delimiters to newString
			set theText to lst as string
		end considering
		set AppleScript's text item delimiters to ASTID
		return theText
	on error eMsg number eNum
		set AppleScript's text item delimiters to ASTID
		error "Can't replaceString: " & eMsg number eNum
	end try
end replaceString