You found this! Honestly, for most of the people, learning a scripting language may be too much; but if you get started using some samples and then ask questions, you can get a lot of help.
I know it has been a while, but is this a possibility? I am interested in seeing a script that creates another file using the results. I have a nice series of scripts (bash, ruby, and LaTex) that take some results, convert that to Multimarkdown → Latex → PDF - > print a determined amount of copies. It sounds complicated, but it is very seamless and produces really nice outputs according to my template. Other than I have to run a search, then copy the results to another file, and then run the script. It has saved me a lot of time (I make about 15 reports a week using this).
Shameless plug 
: I know, Emacs is not for everyone, but this is actually possible with my TaskPaper mode plugin for Emacs:
Tell me the spec, is it:
[FilePath] -> TP3 Document
(A list of file paths (of TaskPaper documents) -> a new document open in TaskPaper)
or
[TP3 Document] -> TP3 Document
(all documents open in TP3 -> a new TP3 document)
And how would you need to specify the sort criteria ?
entering a list of 1-3 (String, Bool) pairs (SortKey|TagName + isAscending) ?
or hardwiring a search spec at the head of a script ?
Anything else ?
PS if we are essentially talking about sorting a merged TP outline, what is the pattern of merging a pair of outlines ?
In particular, if we have a file with various projects, each with some degree of nesting beneath them, how do we decide what subset of items to extract, merge and sort ?
- All top-level items, with descendants, of projects which share names ?
- Or the level of sorting is all top-level projects ?
- Or the top-level children of all projects, leaving the projects themselves in unchanged sequence ?
- Something else ?
I am thinking of something a little bit more simple. I will let you know who I do it, and maybe that will clarify things.
I have this document that I use to keep track of things that people need to do or keep in mind and then I pass the list every week at one of our meetings.
1.- This is the query I do in my source document.
(@type/..* union task///*) except (( //@private/..* union @private///*) union (archive///* union @done))
2.- Then I copy the results into another document that I creatively call "toPrint.taskpaper"
3.- I run a ruby/bash script that does the following,
- Using the Ruby library that Matt Gemmel contributed, I convert the taskpaper document into a temporary multimarkdown file. This could be done in something else other than Ruby, since my only requirement is to change all the projects into a level one header (to just
#) regardless of the hierarchy of the project; and then clean up the taskpaper file of all its tags. - Using the MMD 6.0
LaTex Configability to create templates, the script changes that mdd file into a nice LaTex file that then is processed into a PDF file usingXeLaTex. - Using bash it cleans up the resulting log and aux files created by
XeLaTex, renames the PDF according to the date, and moves the file to a Document repository in a NAS. Since this is something that happens every week, one in our organization can search all the PDF’s by folders. The folders are organized by something likehandOut->YEAR->MONTH->name-based-on-date. - Lastly, using bash and the
lprcommand, the script asks the one who started the script how many copies of the current PDF file are needed and send those to print to a networked printer.
What I would like to do is see is if it would be possible to simplify things even more by having a script that does the query and creates the resulting taskPaper file (overwriting the old one) that is used by my other scripts. That would limited at least one of the steps that take the most time right now (step one and two).
Since I do this at least once a week, that amount of time adds up 
; but I am also thinking to delegate this to someone who is a computer newb. Now, if you need to see some of my source files, so that everything is accomplished in JavaScript/Bash instead of Ruby/Bash I would gladly provide them. Like I said again, they are just simple modifications to the library Matt contributed.
@type/..*
Is that a safely sortable set ? I may well be misunderstanding, but that looks at first sight as if it might create an array in which sub-projects are listed as peers of their parent projects …
More generally, here :
you seemed to have in mind gathering and sorting items from several files, but if I am reading the newer description above correctly, you are working with a single source file, and the issue is more to do with creating a new file, and overwriting an old one ?
My caution about the search you are using is that if your data ever contained any subprojects, it looks at first sight as if there there might be some loss of structure (flattening + duplication) in the report, which makes overwriting feel a little vertiginous …
( PS the earlier goal - generating one report from several files, could, I think, be achieved with Jesse’s standalone Birch-outline API: birch-outline - npm, which I have had a chance to use since those earlier posts, and which does work very well).
Because of how the search queries work, if there is a parent project that had some tasks, even if those tasks are eliminated by the query, the parent project shows up as a result. I don’t remember how I figured it out, but using Paths I can clean up the results. That is why. I think I wrote a post about it somewhere.
Yeah. I formatted the information and pdf template in order to make that extra information irrelevant. I do like to keep that scheme in my original file because of organizational reasons.
Sorry about the separate responds. Using a phone right now.
Currently I could use an example for either. My previous request was kind of solved by just keeping categories of information in the same file. That results in bigger files. If it is not much to ask, could you provide example for both?
Starting with something simple - reports which just sort and group a defined set of items as a flat list (without indented descendants), here is a draft in which you can set some options (at the foot of the script), including the path of the output file to write to.
This version is for the front TaskPaper document in the running app.
I’m not sure how self-explanatory the options at the end are, but note that the orderBy field, needs to be a [ list ] (JS array), even if it contains only one string. The first string specifies primary sort key and direction, and any further strings can specify secondary and tertiary etc sorts.
To avoid grouping, edit the groupBy field to an empty string.
(Test with dummy data from JavaScript mode of Script Editor, or from an Execute JXA action in Keyboard Maestro, etc)
// (c) 2017 Rob Trew
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
// IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
// TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
// OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
// Write sorted and optionally grouped report from TaskPaper to new file
// VER 0.03
// Sample report spec (see end of script):
// {
// outPutFile: '~/Desktop/report.taskpaper',
// itemsInPath: '/@type=project except Archive',
// orderBy: [
// '@priority descending as number',
// '@effort descending as number',
// ],
// groupBy: '@priority as number descending'
// }
(dctOptions => {
'use strict';
// TASKPAPER 3 CONTEXT ---------------------------------------------------
const taskpaperContext = (editor, options) => {
// GENERIC FUNCTIONS -------------------------------------------------
// compare :: a -> a -> Ordering
const compare = (a, b) => a < b ? -1 : (a > b ? 1 : 0);
// comparing :: (a -> b) -> (a -> a -> Ordering)
const comparing = f =>
(x, y) => {
const
a = f(x),
b = f(y);
return a < b ? -1 : (a > b ? 1 : 0);
};
// concatMap :: (a -> [b]) -> [a] -> [b]
const concatMap = (f, xs) =>
xs.length > 0 ? [].concat.apply([], xs.map(f)) : [];
// Handles two or more arguments
// curry :: ((a, b) -> c) -> a -> b -> c
const curry = (f, ...args) => {
const go = xs => xs.length >= f.length ? (f.apply(null, xs)) :
function () {
return go(xs.concat(Array.from(arguments)));
};
return go([].slice.call(args));
};
// drop :: Int -> [a] -> [a]
// drop :: Int -> String -> String
const drop = (n, xs) => xs.slice(n);
// elem :: Eq a => a -> [a] -> Bool
const elem = (x, xs) => xs.indexOf(x) !== -1;
// elems :: Dict -> [a]
const elems = Object.values;
// eq (==) :: Eq a => a -> a -> Bool
const eq = (a, b) => a === b;
// foldl :: (a -> b -> a) -> a -> [b] -> a
const foldl = (f, a, xs) => xs.reduce(f, a);
// groupBy :: (a -> a -> Bool) -> [a] -> [[a]]
const groupBy = (f, xs) => {
const dct = xs.slice(1)
.reduce((a, x) => {
const h = a.active.length > 0 ? a.active[0] : undefined;
return h !== undefined && f(h, x) ? {
active: a.active.concat([x]),
sofar: a.sofar
} : {
active: [x],
sofar: a.sofar.concat([a.active])
};
}, {
active: xs.length > 0 ? [xs[0]] : [],
sofar: []
});
return dct.sofar.concat(
dct.active.length > 0 ? [dct.active] : []
);
};
// intercalate :: String -> [String] -> String
const intercalate = (s, xs) => xs.join(s);
// isNull :: [a] -> Bool
// isNull :: String -> Bool
const isNull = xs =>
Array.isArray(xs) || typeof xs === 'string' ? (
xs.length < 1
) : undefined;
// isPrefixOf :: [a] -> [a] -> Bool
const isPrefixOf = (xs, ys) => {
const pfx = (xs, ys) => xs.length ? (
ys.length ? xs[0] === ys[0] && pfx(
xs.slice(1), ys.slice(1)
) : false
) : true;
return typeof xs !== 'string' ? pfx(xs, ys) : ys.startsWith(xs);
};
// length :: [a] -> Int
const length = xs => xs.length;
// map :: (a -> b) -> [a] -> [b]
const map = (f, xs) => xs.map(f);
// mappendComparing :: [((a -> b), Bool)] -> (a -> a -> Ordering)
const mappendComparing = fboolPairs =>
(x, y) => fboolPairs.reduce(
(ord, [f, b]) => ord !== 0 ? (
ord
) : (
b ? compare(f(x), f(y)) : compare(f(y), f(x))
), 0
);
// sortBy(on(compare,length), xs)
// on :: (b -> b -> c) -> (a -> b) -> a -> a -> c
const on = (f, g) => (a, b) => f(g(a), g(b));
// show :: Int -> a -> Indented String
// show :: a -> String
const show = (...x) =>
JSON.stringify.apply(
null, x.length > 1 ? [x[1], null, x[0]] : x
);
// sortBy :: (a -> a -> Ordering) -> [a] -> [a]
const sortBy = (f, xs) =>
xs.slice()
.sort(f);
// tailDef :: [a] -> [a]
const tailDef = xs => xs.length > 0 ? xs.slice(1) : [];
// toLower :: String -> String
const toLower = s => s.toLowerCase();
// toTitle :: String -> String
const toTitle = s =>
(s.length ? s[0].toUpperCase() + s.slice(1)
.toLowerCase() : '');
// unlines :: [String] -> String
const unlines = xs => xs.join('\n');
// unwords :: [String] -> String
const unwords = xs => xs.join(' ');
// words :: String -> [String]
const words = s => s.split(/\s+/);
// N-ARY SORTS BY ASCENDING OR DESCENDING KEYS -------------------
// Dictionary of sort features -> (getFunction, Ascending?) pair
// customSort :: {key::String, asc::Bool, type::JS Class} ->
// ((Item -> a), Bool)
const customSort = dct => [
propValFn(dct.key, dct.type), dct.asc
];
// Blacklist -> Attributes dictionary -> String of remaining tags
// exceptTags :: [String] -> Dict -> String
const exceptTags = (exclns, dct) =>
unwords(concatMap(
dk => {
const
k = isPrefixOf('data-', dk) ? drop(5, dk) : '',
v = dct[dk];
return (isNull(k) || elem(k, exclns)) ? [] : [
'@' + k + (v !== undefined ? '(' + v + ')' : '')
];
}, Object.keys(dct)
));
// propValFn :: {key::String, asc::Bool, type::JS Class}
// -> (Item -> a)
const propValFn = (k, type) =>
item => ((k !== 'text') ? (
item.getAttribute('data-' + k, type)
) : item.bodyContentString);
const groupValFn = (k, type) =>
group => ((k !== 'text') ? (
group[0].getAttribute('data-' + k, type)
) : group[0].bodyContentString);
// pure :: a -> f a
const pure = x => [x];
// specParse :: String -> (String, Bool, maybe JS Class)
const specParse = s =>
foldl((a, k, i) => {
return k[0] !== '@' ?
elem(
k, ['asc', 'desc', 'ascending', 'descending']
) ? (
a.asc = k[0] === 'a',
a
) : (
elem(k, ['date', 'num', 'number']) ? (
a.type = ((k[0] !== 'd') ? Number : Date),
a
) : a // Unknown word - ignored
) : (
a.key = tailDef(k), // @key
a
);
}, {
key: 'text',
asc: true,
type: undefined
},
words(toLower(s))
);
const
outline = editor.outline,
itemSortFns = map(
ks => customSort(specParse(ks)),
options.orderBy
),
mbGroup = options.groupBy && (options.groupBy.length > 1) ? {
nothing: false,
just: specParse(options.groupBy)
} : {
nothing: true
},
groupProp = mbGroup.nothing ? (
_ => ''
) : propValFn(mbGroup.just.key, mbGroup.just.type),
// Grouping function - single group if no property specified
// groupByPropVal :: [a] -> [[a]]
groupByPropVal = mbGroup.nothing ? (
pure
) : curry(groupBy)(
on(eq, propValFn(mbGroup.just.key, mbGroup.just.type))
);
// itemReport :: Bool -> [String] -> TP3 Item -> String
const itemReport = (blnFull, excludedTags, item) =>
blnFull ? (
item.bodyString
) : item.bodyContentString;
// groupReport :: {key::String, asc::Bool, type:JS type} ->
// [[TP3Item]] -> String
const groupReport = (mbGroup, groups) =>
mbGroup.nothing ? (
unlines(map(
item => itemReport(true, [], item),
groups[0]
))
) : (() => {
const
dct = mbGroup.just,
k = dct.key,
pf = propValFn(k, dct.type);
return unlines(map(gp => {
const v = pf(gp[0]);
return (v !== undefined ? (
toTitle(k) + '(' + v + ')'
) : '(' + k + ' undefined)') + '\n' +
unlines(
map(
x => {
const strType = x.getAttribute(
'data-type'
);
return (strType === 'task' ? (
'- '
) : '') +
x.bodyContentString +
(strType === 'project' ? (
': '
) : ' ') +
exceptTags(
['type', k],
x.attributes
)
},
gp
)
) + '\n';
},
groups
));
})();
// return unlines(map(gp => gp[0].bodyString,
const groups = groupByPropVal(
sortBy(
mappendComparing(itemSortFns),
outline.evaluateItemPath(
options.itemsInPath
)
)
);
return groupReport(mbGroup, mbGroup.nothing ? (
groups
) : sortBy(
mappendComparing([
[groupValFn(mbGroup.just.key, mbGroup.just.type), mbGroup.asc]
]),
groups
));
};
// JXA CONTEXT------------------------------------------------------------
// standardPath :: String -> Path
const standardPath = strPath =>
Path(ObjC.unwrap($(strPath)
.stringByStandardizingPath));
// isNull :: [a] -> Bool
// isNull :: String -> Bool
const isNull = xs =>
Array.isArray(xs) || typeof xs === 'string' ? (
xs.length < 1
) : undefined;
// strip :: String -> String
const strip = s => s.trim();
// writeFileMay :: FilePath -> String -> Maybe FilePath
const writeFileMay = (strPath, strText) => {
const
e = $(),
fullPath = $(strPath)
.stringByStandardizingPath;
return $.NSString.alloc.initWithUTF8String(strText)
.writeToFileAtomicallyEncodingError(
fullPath, false,
$.NSUTF8StringEncoding, e
) ? {
nothing: false,
just: ObjC.unwrap(fullPath)
} : {
nothing: true
};
};
const
tp3 = Application('TaskPaper'),
ds = tp3.documents,
mbd = ds.length > 0 ? {
just: ds.at(0),
nothing: false
} : {
nothing: true
};
const strReport = (mbd.nothing ? (() => {
const nd = new tp3.Document();
return (
ds.push(nd), // Effect
nd // Value
);
})() : mbd.just)
.evaluate({
script: taskpaperContext.toString(),
withOptions: dctOptions
});
// EFFECT – REPORT MAYBE WRITTEN TO FILE ---------------------------------
const mbWritten = isNull(strip(strReport)) ? {
nothing: true
} : writeFileMay(
dctOptions.outPutFile,
strReport
);
// VALUE - MESSAGE RETURNED ----------------------------------------------
return mbWritten.nothing ? (
'Nothing written to ' + standardPath(dctOptions.outPutFile)
) : (
'Written to ' + standardPath(dctOptions.outPutFile) + '\n\n' +
strReport
);
})({
outPutFile: '~/Desktop/report.taskpaper',
itemsInPath: '/@type=project except Archive',
orderBy: [
'@priority descending as number',
'@effort descending as number',
],
groupBy: '@priority as number descending'
});
Thank you very much. I just have a question. Is there a particular reason why all the @priority tags are changed into @t in the report?
Now, is it possible to simplify this script even a little bit more and make this into something that just runs a query and makes a new file with the results. In that way I can first understand what is going on and how you can create new files. Then I can go on to try to figure out how the sorting and ordering is taking place. There is no rush. so just whenever you get a chance.
Again, Thank you very much. Have. good weekend.
Is there a particular reason why all the @priority tags are changed into @t in the report?
Thanks – fixed above in 0.02
Simpler version
Conceptually, there are, as you know, two bits of code in all these scripts:
- A single function for TaskPaper (possibly enclosing other functions inside it), which JXA converts into a string, and submits to the other Javascript interpreter, embedded inside TaskPaper
- The JXA code.
The key point here is that the TaskPaper code evaluation can return a value to JXA. e.g.
const strReport = doc.evaluate({
script: taskpaperContext.toString(),
withOptions: dctOptions
});
Once JXA has received that return value, it can write it out to a file. For this stage, you need a function like:
// writeFile :: FilePath -> String -> IO ()
const writeFile = (strPath, strText) =>
$.NSString.alloc.initWithUTF8String(strText)
.writeToFileAtomicallyEncodingError(
$(strPath)
.stringByStandardizingPath, false,
$.NSUTF8StringEncoding, null
);
or, if you are feeling more cautious, or interested to trap failure conditions:
// writeFileMay :: FilePath -> String -> Maybe FilePath
const writeFileMay = (strPath, strText) => {
const
e = $(),
fullPath = $(strPath)
.stringByStandardizingPath;
return $.NSString.alloc.initWithUTF8String(strText)
.writeToFileAtomicallyEncodingError(
fullPath, false,
$.NSUTF8StringEncoding, e
) ? {
nothing: false,
just: ObjC.unwrap(fullPath)
} : {
nothing: true
};
};
Have to go out now, but I’ll aim to sketch you a minimal sorter for this pattern later.
1 Like
Vanilla and option-free:
(() => {
'use strict';
// TASKPAPER 3 CONTEXT ---------------------------------------------------
const taskpaperContext = editor => {
// comparing :: (a -> b) -> (a -> a -> Ordering)
const comparing = f =>
(x, y) => {
const
a = f(x),
b = f(y);
return a < b ? -1 : (a > b ? 1 : 0);
};
return editor.outline.evaluateItemPath(
'/@type=project except Archive'
)
.sort(comparing(x => x.getAttribute('data-priority', Number)))
.map(x => x.bodyString)
.join('\n');
};
// JXA CONTEXT------------------------------------------------------------
// writeFile :: FilePath -> String -> IO ()
const writeFile = (strPath, strText) =>
$.NSString.alloc.initWithUTF8String(strText)
.writeToFileAtomicallyEncodingError(
$(strPath)
.stringByStandardizingPath, false,
$.NSUTF8StringEncoding, null
);
const
ds = Application('TaskPaper')
.documents;
const strReturnValue = ds.length > 0 ? ds.at(0)
.evaluate({
script: taskpaperContext.toString()
}) : 'No document open in TaskPaper 3';
return (
writeFile('~/Desktop/test.taskpaper', strReturnValue),
strReturnValue
);
})();
1 Like
This is amazing. Now I am going to be googling how to run Ruby Scripts from JavaScript. Really. This works perfectly. I took your code and simplified a little bit more. I eliminated the “sort” option since part of what I wanted to see how to do was to run a query and then write the results into a new file. Then I could start to process the rest.
Here is your code minus the sort part.
(() => {
'use strict';
// ------------------------- TASKPAPER 3 CONTEXT --------------------------
const taskpaperContext = editor => {
return editor.outline.evaluateItemPath(
'(@type/..* union task///*) except (( //@private/..* union @private///*) union (archive///* union @done))'
)
.map(x => x.bodyString)
.join('\n');
};
// ---------------------------- JXA CONTEXT ------------------------------
// writeFile :: FilePath -> String -> IO ()
const writeFile = (strPath, strText) =>
$.NSString.alloc.initWithUTF8String(strText)
.writeToFileAtomicallyEncodingError(
$(strPath)
.stringByStandardizingPath, false,
$.NSUTF8StringEncoding, null
);
const
ds = Application('TaskPaper')
.documents;
const strReturnValue = ds.length > 0 ? ds.at(0)
.evaluate({
script: taskpaperContext.toString()
}) : 'No document open in TaskPaper 3';
return (
writeFile('~/Desktop/test.taskpaper', strReturnValue),
strReturnValue
);
})();
In my particular case, I don’t need to see the position of the projects in the document tree (represented with indentations in TaskPaper), but it might be good for the code to keep that. Is that something easy to solve?
I think that item.getattribute will let you read the value of the built-in attribute indent (which doesn’t have a ‘data-’ prefix)
Or, perhaps more directly, you can use the item.depth property
Thank you. It is hard to figure out where to find documentation to understand the code. I think that I am starting to put the puzzle together.
Now, this is a question to @jessegrosjean
.map(x => x.bodyString)
Why is the indentation not included in the bodyString method? Is there a way to do this easily? Can you extend the code so that one could just get some of the information (what type of item this is, the text, and the tabs, but ignore the tags, etc.)?
Why is the indentation not included in the bodyString method? Is there a way to do this easily?
The API is very complete – all of the information is there and flexibly usable.
If you wanted to preserve indentation, for example, one way would be to add tabs or spaces * (Item.depth - 1):
map(x => concat(replicate(x.depth - 1, '\t')) + x.bodyString)
As in
(() => {
'use strict';
// Vanilla 0.2 (with indents)
// TASKPAPER 3 CONTEXT ---------------------------------------------------
const taskpaperContext = editor => {
// comparing :: (a -> b) -> (a -> a -> Ordering)
const comparing = f =>
(x, y) => {
const
a = f(x),
b = f(y);
return a < b ? -1 : (a > b ? 1 : 0);
};
// concat :: [[a]] -> [a]
// concat :: [String] -> String
const concat = xs =>
xs.length > 0 ? (() => {
const unit = typeof xs[0] === 'string' ? '' : [];
return unit.concat.apply(unit, xs);
})() : [];
// replicate :: Int -> a -> [a]
const replicate = (n, x) =>
Array.from({
length: n
}, () => x);
return editor.outline.evaluateItemPath(
'//@type=project except Archive'
)
.sort(comparing(x => x.getAttribute('data-priority', Number)))
.map(x => concat(replicate(x.depth - 1, '\t')) + x.bodyString)
.join('\n');
};
// JXA CONTEXT------------------------------------------------------------
// writeFile :: FilePath -> String -> IO ()
const writeFile = (strPath, strText) =>
$.NSString.alloc.initWithUTF8String(strText)
.writeToFileAtomicallyEncodingError(
$(strPath)
.stringByStandardizingPath, false,
$.NSUTF8StringEncoding, null
);
const
ds = Application('TaskPaper')
.documents;
const strReturnValue = ds.length > 0 ? ds.at(0)
.evaluate({
script: taskpaperContext.toString()
}) : 'No document open in TaskPaper 3';
return (
writeFile('~/Desktop/test.taskpaper', strReturnValue),
strReturnValue
);
})();
1 Like
Why is the indentation not included in the bodyString method?
I can’t speak for Jesse, but as a scripter I can tell you that it would often prove a nuisance if it were included as a literal string rather than as a separate indent/depth number :- )
(When we didn’t need the leading whitespace we would need to manually remove it ourselves, and when we did need to know its length we would need to manually measure it ourselves.
The API gives us the line pre-analysed, with a number for the indent, and a string for the text)
1 Like