c.p.brown
2nd of July 2025


2020s
TXT
gifded:gtk4::vala:
screenie
Bash together a process by mixing Python, Rebol, text or Shellscript operators... whatever works.

Used for both commercial and personal scripting projects.
This gets a lot of use, pretty happy with it.

Source is here.
Note there is no license to use this software, it exists here for my own use, should I need to obtain it via internet.

Instructions for Linux:
  • extract && cd to gifded
  • install vala and gtksourceview5-devel
  • copy install/.local/share/gtksourceview-5 to /.local/share/
  • compile using clear && valac gifded.vala --pkg gtk4 --pkg gtksourceview-5 -X -lm
  • install any other dependencies it complains about, and try again
  • run ./gifded
  • if it looks like ass, install adw-gtk3-theme and enable via gnome-tweaks
  • if it still looks like ass, SOLS: its a backward compatibility issue (being addressed in the sequel via hardcoded theme)

Currently being replaced by a branching editor, main differences are:
  • customizable ui layout in desktop mode
  • nodes instead of list
  • script node supports compilation where applicable
  • script node supports more languages (c, cpp, common lisp, rust, spark, vala)
  • option for script node to use a compiled binary if it exists
  • fast output viewer, scroll through millions of lines without lag
  • hardcoded QOL nodes, mostly for handling 2d data and i/o
  • hardcoded theme to use standardized branding, early preview below
screenie

fulltardie:gtk4::vala::wip:
screenie

Forecast complex recurring transaction rules.

Used to keep track of a large number of recurring transactions.
Also great for quickly testing future scenarios.

Currently being re-written, changes are:
  • search & filter lists
  • faster lists and calendar
  • graph: pies, nested circles, sankey
  • graph export to png or pdf
  • Gnome notification integration
  • hardcoded theme using consistent branding, early preview below
preview

r21:gtk3::red:
screenie

Used to manage my 2021 site



dramaqueen:gtk3::red:
screenie
Site builder with T&C configurator.

Used for commerical projects, since replaced by a much faster commandline version written in Vala.
reexsa:gtk3::red:
screenie
Interface for testing Red-Parse.

Red [ needs 'view ] ;; interactively test parse rules ;; by c.p.brown 2021 ;; ;; attempt [] may freeze the script, uncomment if its a problem marg: 10 tabh: 0 ;; height of the statusbar (none atm) nudgeu: func [] [ uu/offset/x: 0 uu/offset/y: min (max 200 uu/offset/y) 600 uu/size/x: tp/size/x vv/offset/y: uu/offset/y + 10 vv/size/y: tp/size/y - ((uu/offset/y + 10) + tabh) maa/offset/y: 0 maa/size/x: uu/size/x mbb/offset/y: uu/offset/y + 10 mcc/offset/y: uu/offset/y + 10 maa/size/y: uu/offset/y mbb/size/y: vv/size/y mcc/size/y: vv/size/y maah/offset/y: 0 mbbh/offset/y: 0 mcch/offset/y: 0 maap/offset/y: maah/size/y + marg mbbp/offset/y: mbbh/size/y + marg mccp/offset/y: mcch/size/y + marg maap/size/y: maa/size/y - (maah/size/y + (2 * marg)) mbbp/size/y: mbb/size/y - (mbbh/size/y + (2 * marg)) mccp/size/y: mcc/size/y - (mcch/size/y + (2 * marg)) uu/draw: compose/deep [ pen off fill-pen 100.100.100 circle (to-pair compose/deep [(to-integer (uu/size/x * 0.5) - 10) 5]) 2 2 circle (to-pair compose/deep [(to-integer (uu/size/x * 0.5)) 5]) 2 2 circle (to-pair compose/deep [(to-integer (uu/size/x * 0.5) + 10) 5]) 2 2 ] ] nudgev: func [] [ vv/offset/y: uu/offset/y + 10 vv/offset/x: min (max 100 vv/offset/x) (tp/size/x - 100) if vv/offset/x < 100 [ return 0 ] maa/offset/x: 0 mbb/offset/x: 0 mcc/offset/x: vv/offset/x + 10 mbb/size/x: vv/offset/x mcc/size/x: (tp/size/x - mcc/offset/x) maah/offset/x: 0 mbbh/offset/x: 0 mcch/offset/x: 0 maah/size/x: maa/size/x mbbh/size/x: mbb/size/x mcch/size/x: mcc/size/x maap/offset/x: marg mbbp/offset/x: marg mccp/offset/x: marg maap/size/x: maah/size/x - (marg + marg + 1) mbbp/size/x: mbbh/size/x - (marg + marg) mccp/size/x: mcch/size/x - (marg + marg + 1) vv/draw: compose/deep [ pen off fill-pen 100.100.100 circle (to-pair compose/deep [5 (to-integer (vv/size/y * 0.5) - 10)]) 2 2 circle (to-pair compose/deep [5 (to-integer (vv/size/y * 0.5))]) 2 2 circle (to-pair compose/deep [5 (to-integer (vv/size/y * 0.5) + 10)]) 2 2 ] ] view/tight/flags/options [ title "reexsa" below hh: panel 800x55 35.35.35 [ parsers: drop-list 200x30 font-name "consolas" font-size 10 font-color 180.180.180 bold data (collect [foreach file read %./ [ if (find (to-string file) ".parse") [keep rejoin ["./" (to-string file)]] ]]) on-change [ pfn: copy parsers/data/(parsers/selected) pn: "" parse pfn [ thru "./" copy pn to ".parse" ] parsername/text: pn pset: load to-file parsers/data/(parsers/selected) maap/text: (reduce pset/1) mbbp/text: (reduce pset/2) maap/font/color: 255.0.0 s: copy mbbp/text attempt [ do maap/text mccp/text: s maap/font/color: 128.255.128 ] ] pad 10x0 parsername: field 200x30 font-name "consolas" font-size 10 font-color 180.180.180 bold pnew: button 80x34 "save" font-name "consolas" font-size 10 font-color 180.180.180 bold [ if (parsername/text <> "") and (parsername/text <> none) [ newparsername: rejoin [ "./" parsername/text ".parse" ] write to-file newparsername ( reduce [ maap/text mbbp/text ]) clear parsers/data parsers/data: (collect [foreach file read %./ [ if (find (to-string file) ".parse") [keep rejoin ["./" (to-string file)]] ]]) parsers/selected: index? find parsers/data newparsername ] ] pad 10x0 psave: button 160x34 "save output" font-name "consolas" font-size 10 font-color 180.180.180 bold [ if (parsername/text <> "") and (parsername/text <> none) [ op: request-dir unless none? op [ write to-file rejoin [ op parsername "_output.txt" ] mccp/text ] ] ] ] tp: panel 800x745 [ below maa: panel 800x300 40.40.40 [ maah: panel 45.45.45 800x55 [ text 200x30 "parse" font-name "consolas" font-size 24 font-color 80.80.80 bold ] maap: area 800x300 40.40.40 font-name "consolas" font-size 12 font-color 128.255.128 bold with [ text: {parse s [ any [ to "^^/* " change "^^/* " "^^/<li>" pre: [ to "^^/" change "^^/" "</li>^^/" | to end change end "</li>" end ] :pre ] ]^/parse s [ any [ to "^^/<li>" not "</li>^^/<li>" change "^^/<li>" "<ul>^^/<li>" ] ]} ] on-change [ face/font/color: 255.0.0 s: copy mbbp/text attempt [ do maap/text mccp/text: s face/font/color: 128.255.128 ] ] ] uu: panel 800x10 30.30.30 loose [] on-drag [ nudgeu ] across mbb: panel 390x400 40.40.40 [ mbbh: panel 390x55 45.45.45 [ text 200x30 "source" font-name "consolas" font-size 24 font-color 80.80.80 bold ] mbbp: area 390x300 40.40.40 font-name "consolas" font-size 12 font-color 128.255.255 bold with [ text: {a list:^/* one^/* two^/* three^/end of * list^/another list^/* AAA^/* BBB^/end of the other list} ;;*/ <- emacs red-mode is screwy atm ] on-change [ face/font/color: 255.0.0 s: copy mbbp/text attempt [ do maap/text mccp/text: s face/font/color: 128.255.128 ] ] ] vv: panel 10x390 30.30.30 loose [] on-drag [ nudgev ] mcc: panel 390x400 40.40.40 [ mcch: panel 390x55 45.45.45 [ text 200x30 "result" font-name "consolas" font-size 24 font-color 80.80.80 bold ] mccp: area 390x345 40.40.40 font-name "consolas" font-size 12 font-color 255.255.128 bold ] ] do [ if exists? %./default.parse [ pset: load %./default.parse maap/text: (reduce pset/1) mbbp/text: (reduce pset/2) parsername/text: "default" parsers/selected: index? find parsers/data "./default.parse" maap/font/color: 255.0.0 s: copy mbbp/text attempt [ do maap/text mccp/text: s maap/font/color: 128.255.128 ] ] psave/offset/x: tp/size/x - (psave/size/x + marg) pnew/offset/x: psave/offset/x - (pnew/size/x + marg) parsername/offset/x: (parsers/offset/x + parsers/size/x + marg) parsername/size/x: pnew/offset/x - (parsers/size/x + (4 * marg)) nudgeu nudgev ] ] [ resize ] [ actors: object [ on-resizing: function [ face event ] [ if face/size/x > 500 [ if face/size/y > (uu/offset/y + 200) [ hh/size/x: face/size/x tp/size: face/size - 0x55 psave/offset/x: face/size/x - (psave/size/x + marg) pnew/offset/x: psave/offset/x - (pnew/size/x + marg) parsername/offset/x: (parsers/offset/x + parsers/size/x + marg) parsername/size/x: pnew/offset/x - (parsers/size/x + (4 * marg)) nudgeu nudgev ] ] ] ] ]
REF
HEXTILE:hou::vex:

Hextile gradient inline vex (COPs).
Derivative work of various sources, most helpful was Procedural Graphics Tutorial: Hexagon Effects by Andrew Hung.
Used as a transition input for another effect.

// hextiles // this is a derivative work, from many conflicting sources // uses in-situ colormap for the control gradient, as its uvs are derived by the hex coords // code is unrolled for clarity float sqa = 1.0; float sqb = sqrt(3.0); float uu = $X; float vv = $Y; vector2 uv = set(uu,vv); float squ = (sqb * 0.5); float sqv = 0.5; float hxca = floor(uu / sqa) + 0.5; float hxcb = floor(vv / sqb) + 0.5; float hxcc = floor((uu - 0.5) / sqa) + 0.5; float hxcd = floor((vv - 1.0) / sqb) + 0.5; float ofsa = (uu - (hxca * sqa)); float ofsb = (vv - (hxcb * sqb)); float ofsc = (uu - ((hxcc + 0.5) * sqa)); float ofsd = (vv - ((hxcd + 0.5) * sqb)); vector2 aa = set(ofsa,ofsb); vector2 bb = set(ofsc,ofsd); float da = length(aa); float db = length(bb); float dsta = ofsc; float dstb = ofsd; float dstc = hxcc * 0.1; float dstd = hxcd * 0.5; float idx = hxcc * 1.0; float idy = hxcd * 1.0; if (da < db) { dsta = ofsa; dstb = ofsb; dstc = hxca; dstd = hxcb; idx = hxca - 1.0; idy = hxcb - 1.0; } float adsta = abs(dsta); float adstb = abs(dstb); float adstc = abs(dstc); float adstd = abs(dstd); vector2 sqh = set(0.5,(sqb * 0.5)); vector2 sqq = set(sqa,sqb); vector2 dsva = set(adsta,adstb); vector2 dsvb = set(adstc,adstd); float gg = max(dot(sqh,dsva),adsta); float mm = $MM; float cc = 0.9; float gsu = clamp((idx / $WW),0.0,1.0); $width = gsu; float gsv = clamp((idy / $WW),0.0,1.0); $height = gsv; vector4 pxl = colormap($IMG,gsu,gsv); $heximg = pxl; float bw = (gsu + pxl.x); $dist = clamp(((gg - (pxl.y + $MM)) / 0.05),0.0,1.0); $tile = abs(idy);
SDFBOXMASK:hou::vex:

Rectangular sdf mask inline vex (COPs).
This is a vex translation of a polygon shader by Inigo Quilez here.
Used for masking with rounded corners.

// vex translation of this shader by Inigo Quilez: // https://www.shadertoy.com/view/wdBXRW float uu = $X; float vv = $Y; vector2 p = set(uu,vv); float hrr = $RR * 1.0; vector2 tl = set($XEA+hrr,$YEA-hrr); vector2 bl = set($XEA+hrr,$YEB+hrr); vector2 br = set($XEB-hrr,$YEB+hrr); vector2 tr = set($XEB-hrr,$YEA-hrr); vector2 v[] = array(tl,bl,br,tr); int num = 4; float d = dot((p - v[0]),(p - v[0])); float s = 1.0; for( int i=0, j=num-1; i<num; j=i, i++ ) { // distance vector2 e = v[j] - v[i]; vector2 w = p - v[i]; vector2 b = w - (e * clamp((dot(w,e) / dot(e,e)), 0.0, 1.0)); d = min(d, dot(b,b)); //winding number from http://geomalgorithms.com/a03-_inclusion.html vector cond = set( (p.y >= v[i].y), (p.y < v[j].y), ((e.x * w.y) > (e.y * w.x))); if (((cond[0] == 1) && (cond[1] == 1) && (cond[2] == 1)) || ((cond[0] == 0) && (cond[1] == 0) && (cond[2] == 0))) { s = -s; } } float dist = (s * sqrt(d)); //float dist = -(sqrt(d.x) * sign(d.y)); dist = dist - $RR; dist = clamp(dist,0.0,1.0); $OUT = 1.0 - clamp((dist / 0.005),0.0,1.0);
VALA TESTS:vala:

Scratchpad of ideas/techniques tested while learning Vala.

Code is incomplete, incorrect, ill-conceived, unsafe, etc..

1DARRAYS:vala:
Use a 1d array + colcount for 2d data, as a workaround for non resizable native 2d arrays.

  • row at index is (index / colcount)
  • col at index is (index % colcount)
  • index at (row,col) is ((row * colcount) + col)
  • rowcount is (length / colcount)

pros: can resize in situ; no need to duplicate data
cons: extra work

string printstringrowcol (int c, string[] s) { string o = "{ "; for (int i = 0; i < (s.length - 1); i++) { o = (o + s[i]); if (((i+1) % c) == 0) { o = o + "\n "; } else { o = o + ";"; } } o = (o + s[(s.length - 1)] + " }\n"); return o; } void main() { string[] s = { "0", "1", "2", "3", "1", "A", "B", "C", "2", "X", "Y", "Z" }; // col 0 1 2 3 0 1 2 3 0 1 2 3 // row 0 0 0 0 1 1 1 1 2 2 2 2 // index 0 1 2 3 4 5 6 7 8 9 10 11 int colcount = 4; // note: count, not index int rowcount = (s.length / colcount); print("s is \n%s",printstringrowcol(colcount,s)); int row = 0; int col = 3; int idx = (col+(row*colcount)); print("index of row[%d] and col[%d] is %d \"%s\"\n",row,col,idx,s[idx]); // get row and column from index idx = 3; row = idx / colcount; col = idx % colcount; print("(row,col) at index %d is (%d,%d)\n",idx,row,col); // remove col 1 col = 1; int lastcol = colcount - 1; int n = 0; for (int i = col; i < (s.length - rowcount); i++) { if (((i-col) % lastcol) == 0) { n += 1; } s[i] = s[i+n]; } s.resize(s.length - rowcount); colcount -= 1; print("s without col[%d] is \n%s",1,printstringrowcol(colcount,s)); // get index from row and column row = 2; col = 1; idx = (col+(row*colcount)); print("index of row[%d] and col[%d] is %d \"%s\"\n",row,col,idx,s[idx]); row = idx / colcount; col = idx % colcount; print("(row,col) at index %d is (%d,%d)\n",idx,row,col); // test int[] ia = {0, 1, 2, 3, 4, 5}; ia = ia + 1; foreach (int i in ia) { print("%d\n",i); } }
#+RESULTS:
s is { 0;1;2;3 1;A;B;C 2;X;Y;Z } index of row[0] and col[3] is 3 "3" (row,col) at index 3 is (0,3) s without col[1] is { 0;2;3 1;B;C 2;Y;Z } index of row[2] and col[1] is 7 "Y" (row,col) at index 7 is (2,1) 0 1 2 3 4 5 1
ASCIISORT:busted::vala:

This is being replaced by RECURSIVERADIX, will be removed.

// ascii sort // by c.p.brown 2025 // // uses quicksort after partitioning row/line indices // // test data found here: ttps://introcs.cs.princeton.edu/java/data/leipzig/leipzig1m.txt // and here: https://www.dol.gov/agencies/eta/foreign-labor/performance GLib.Mutex mutex; string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } struct packet { public int[] items; } public struct retardedpacket { int[] items; } packet[] packseq (int ind, int[] ix, int c, bool q, int y, int z, int u, string m, double[] bx, int fx) { // x = items to package // q = use items if true, use items count if false, eg {2,12,34,43} vs {0,1,2,3} // y = max packets, usually thread-count // z = nth item to package, eg {0,1,2,3,4,5} nth 2 is {0,2,4} // u = offset items, eg {0,1,2,3,4,5} offset 2 is {2,3,4,5}, {0,1,2,3,4,5} nth 2 offset 1 is {1,3,5} // m = mode "" is uniform packets, "BIAS" is small to large packets packet[] opk = {}; int[] srcitems = ix; int maxpackets = y; int nthsrc = z; // nth int srcoffset = u; // offset int srclen = c; int[] sampleditems = {}; int[] itempacket = {}; if (q) { srclen = srcitems.length; for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += srcitems[j]; } } else { for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += j; } } if (sampleditems.length > maxpackets) { if (m == "BIAS") { int[] qb = {}; int dl = sampleditems.length; for (int n = 0; n < dl; n++) { double x = bx[n]; double s = 0.8; double g = 0.0; double f = 1.0; if (x < 1.5) { g = ((x * f) / ((((1.0 / s) - 2.0) * (1.0 - (f * x))) + 1.0)) / f; } else { g = ((((x * 2.0) - 1.0) / ((((1.0 / (1.0 - s)) - 2.0) * (1.0 - ( (2.0 * x) - 1.0))) + 1.0)) / 2.0) + 0.5; } double w = maxpackets - 1; double h = (g * w); int bar = ((int) h); //string plot = ""; //for (int j = 0; j < bar; j++) { // plot = plot + "█"; //} itempacket += bar; //print("%02d | %0.2f | %0.4f\t| ascii %0.2d\t (%s)\t | bx %0.2f | %02d | %s\n",n,x,g,sampleditems[n],((char) sampleditems[n]).to_string(),bx[n],bar,plot); qb += bar; } //packet[] op = {}; for (int i = 0; i < maxpackets; i++) { opk += new packet(); } for (int r = 0; r < sampleditems.length; r++) { for (int i = 0; i < maxpackets; i++) { if (itempacket[r] == i) { opk[i].items += sampleditems[r]; } } } return opk; } if (m == "FIT") { //packet[] o = {}; for (int i = 0; i < maxpackets; i++) { opk += new packet(); } int accumulatedcount = 0; int totalcount = fx; int remainingcount = totalcount; int maxcount = ((int) Math.ceil((fx * 1.0) / (maxpackets * 1.0))) ; //print("totalcount (%d) / maxpackets (%d) is %d\n",fx,maxpackets,maxcount); //print("sampleditems.length is %d\n",sampleditems.length); int i = 0; for (int r = 0; r < sampleditems.length; r++) { int remainingpackets = maxpackets - i; int charcount = ((int) bx[r]); int countdiff = maxcount - charcount; remainingcount = remainingcount - charcount; //print("[%03d] packet count is %05d, maxcount is %05d, remaining count is %05d,",r,(accumulatedcount + charcount),maxcount, remainingcount); if (accumulatedcount < maxcount) { if ((charcount > (maxcount - accumulatedcount)) && (i > 0)) { i++; opk[i].items += sampleditems[r]; accumulatedcount = 0; } else { opk[i].items += sampleditems[r]; } } else { if (i < (maxpackets - 1)) { i++; opk[i].items += sampleditems[r]; } else { opk[i].items += sampleditems[r]; } accumulatedcount = 0; } if (i < (maxpackets - 1)) { maxcount = ((int) Math.ceil((remainingcount * 1.0) / (remainingpackets * 1.0))); } else { maxcount = totalcount; } //print(" adding char[%03d] %03d (%s) to seg[%d]\n",r,sampleditems[r],((char) sampleditems[r]).to_string(),i); accumulatedcount += charcount; } return opk; } if (m == "") { int itemsperpacket = (sampleditems.length / maxpackets); double neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); int mxn = 0; while (neededpackets > maxpackets) { if (mxn > 1000000) { //packet[] o = new packet[0]; return opk; } itemsperpacket += 1; neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); mxn += 1; } int np = (int) neededpackets; int dregs = sampleditems.length - (itemsperpacket * ((int) neededpackets)); int h = 0; if (dregs > 0) { h = 1; } if ((np + h) > 0) { opk.resize(np+h); //packet[] o = new packet[(np+h)]; int f = 0; for (int i = 0; i < np; i++) { opk[i] = new packet(); for (int t = 0; t < itemsperpacket; t++) { opk[i].items += sampleditems[f]; f += 1; } } if (dregs > 0) { opk[np] = new packet(); for (int t = 0; t < dregs; t++) { opk[np].items += sampleditems[f]; f += 1; } } return opk; } } } else { // one item per packet //packet[] o = {}; for (int k = 0; k < sampleditems.length; k++) { packet oo = new packet(); oo.items += sampleditems[k]; opk += oo; } return opk; } packet[] ov = new packet[0]; return ov; } string printintarray (int[] s, int f) { string[] ii = {"", "\"", "{", "[", "[", "| ", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%d".printf(s[i]) + dd[f]); } o = (o + "%d".printf(s[s.length - 1]) + oo[f]); return o; } string printstringarray (string[] s, int f) { // 0 1 2 3 4 5 6 7 8 string[] ii = {"", "\"", "{", "[", "[", "| ", "", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n", ",\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%s".printf(s[i]) + dd[f]); } o = (o + "%s".printf(s[s.length - 1]) + oo[f]); return o; } int[] numchars; int[] prenums; int strintcmp (string a, string b) { if ((a.length > 1) && (b.length > 1)) { int ca = ((int) a.get_char(a.index_of_nth_char(0))); int cb = ((int) b.get_char(b.index_of_nth_char(0))); int nn = 0; int na = 1; while (ca in prenums) { if (na >= a.length) { nn = 1; break; } ca = ((int) a.get_char(a.index_of_nth_char(na))); na += 1; } int nb = 1; while (cb in prenums) { if (nb >= b.length) { nn = 1; break; } cb = ((int) b.get_char(b.index_of_nth_char(nb))); nb += 1; } int ea = na; int eb = nb; if (nn == 0) { int cea = ((int) a.get_char(a.index_of_nth_char(na))); int ceb = ((int) b.get_char(b.index_of_nth_char(nb))); while (cea in numchars) { if (ea >= a.length) { break; } cea = ((int) a.get_char(a.index_of_nth_char(ea))); ea += 1; } while (ceb in numchars) { if (eb >= b.length) { break; } ceb = ((int) b.get_char(b.index_of_nth_char(eb))); eb += 1; } } if ((nn == 0) && (ca in numchars) && (cb in numchars)) { StringBuilder sba = new StringBuilder(""); StringBuilder sbb = new StringBuilder(""); if ((ea > na) && (ea < (a.length - 1))) { sba.append(a.substring(na,(ea-na))); } else { sba.append(a.substring(na)); } if ((eb > nb) && (eb < (b.length - 1))) { sbb.append(b.substring(nb,(eb-nb))); } else { sbb.append(b.substring(nb)); } //sba.replace("\"","",1); //sba.replace("$","",1); sba.replace("/","",2); // omit the following for fucky EU numbers where they use commas as decimal-points sba.replace(",","",0); sba.replace("\'","",0); //sbb.replace("\"","",1); //sbb.replace("$","",1); sbb.replace("/","",2); sbb.replace(",","",0); sbb.replace("\'","",0); if ((sba.str.length > 0) && (sbb.str.length > 0)) { //print("%s - %s...\n",sba.str,sbb.str); int dd = ((int) (double.parse(sba.str) - double.parse(sbb.str))); if ( (dd == 0) && ((ea > na) && (ea < (a.length - 1))) && ((eb > nb) && (eb < (b.length - 1))) ) { return strcmp(a.substring(ea),b.substring(eb)); } return dd; } } } return strcmp(a,b); } packet[] binstrings (string[] s, packet[] q, int c, int cc) { packet[] o = new packet[q.length]; int r = 0; unichar[] numprefix = {'$','+','-'}; for (int i = c; i < s.length; i += cc) { unichar ff = s[i].get_char(s[i].index_of_nth_char(0)); if (s[i].char_count() > 1) { unichar ffb = s[i].get_char(s[i].index_of_nth_char(1)); if (ffb.isdigit()) { // put row number into a number bin if it looks like a number with prefix, eg: +2, $2, -2 if (ff in numprefix) { ff = ffb; } } } int fx = ((int) ff); int filed = 0; for (int j = 0; j < q.length; j++) { if (fx >= q[j].items[0]) { if (fx <= q[j].items[(q[j].items.length - 1)]) { o[j].items += r; filed = 1; break; } } } if (filed == 0) { o[0].items += r; } r += 1; } return o; } void rawquicksortonedee (ref string[] dat, int s, int e) { if ((e - s) > 0) { int p = s; //((s+e)>>1); int i = s; int j = e; while(i <= j) { while (strcmp(dat[i],dat[p]) < 0) { i += 1; } while (strcmp(dat[j],dat[p]) > 0) { j -= 1; } if (i <= j) { string t = dat[i]; dat[i] = dat[j]; dat[j] = t; i += 1; j -= 1; } } rawquicksortonedee(ref dat,s,j); rawquicksortonedee(ref dat,i,e); } } void quicksortindexbycol (ref int[] dxs, int c, int cc, int s, int e, string[] dat) { if ((e - s) > 0) { int p = s; int i = s; int j = e; while(i <= j) { while (strcmp(dat[(dxs[i]*cc)+c],dat[(dxs[p]*cc)+c]) < 0) { i += 1; } while (strcmp(dat[(dxs[j]*cc)+c],dat[(dxs[p]*cc)+c]) > 0) { j -= 1; } if (i <= j) { int t = dxs[i]; dxs[i] = dxs[j]; dxs[j] = t; i += 1; j -= 1; } } quicksortindexbycol(ref dxs,c,cc,s,j,dat); quicksortindexbycol(ref dxs,c,cc,i,e,dat); } } void quicksortstringsbycol (ref string[] dat, int ct, int c, int cc, int s, int e) { // note: start (s) and end (e) are ROWS, from-and-including, to-and-including // ct is column type, 0 bool, 1 int, 2 double, 3 string, 4 datetime if ((e - s) > 0) { int p = (s+((e-s)/2)); //((s+e)>>1); int i = s; int j = e; //print("first is %d, last is %d, mid is %d\n",i,j,p); //print("column is %d, columncount is %d\n",c,cc); //print("comparing dat[%d] (%s) with dat[%d] (%s) : %d\n",((i*cc)+c),dat[((i*cc)+c)],((p*cc)+c),dat[((p*cc)+c)],strcmp(dat[(i*cc)+c],dat[(p*cc)+c])); while(i <= j) { if (ct == 1) { while (int.parse(dat[(i*cc)+c]) < int.parse(dat[(p*cc)+c])) { i += 1; } while (int.parse(dat[(j*cc)+c]) > int.parse(dat[(p*cc)+c])) { j -= 1; } } else { while (strcmp(dat[(i*cc)+c],dat[(p*cc)+c]) < 0) { i += 1; } while (strcmp(dat[(j*cc)+c],dat[(p*cc)+c]) > 0) { j -= 1; } } if (i <= j) { string tmp = ""; int n = 0; for (int g = (j * cc); g < ((j * cc) + cc); g++) { tmp = dat[g]; dat[g] = dat[((i * cc) + n)]; dat[((i * cc) + n)] = tmp; n += 1; } i += 1; j -= 1; } } //print("spawn quicksorts from %d to %d, then again from %d to %d\n",s,j,i,e); if (j < i) { quicksortstringsbycol(ref dat,ct,c,cc,s,j); quicksortstringsbycol(ref dat,ct,c,cc,i,e); } } } void getchardists(ref int[] ch, string[] s, int c, int cc) { for (int i = c; i < s.length; i += cc) { unichar ff = s[i].get_char(0); int fx = ((int) ff); if (fx < 127) { // file char count under its char index at col 1 of 2 ch[(fx*3)+2] += 1; ch[(fx*3)+1] = fx; } else { // char is out of range, file it under 1st item c[0] ch[2] += 1; ch[1] = -1; } } } string printunrolledarray (int ind, string[] dat, int dsz, bool h, bool rnm, bool cnm, int m) { string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string spc = " "; bool domask = (m > -1); string dlm = " | "; int rcols = dsz; int rowcount = dat.length / dsz; if (rnm) { rcols = dsz+1; } int chc = 0; if (cnm) { chc = "%d".printf((dat.length - 1)).char_count(); } int[] cw = new int[rcols]; for (int i = 0; i < cw.length; i++) { cw[i] = 0; } if (rnm) { cw[0] = "%d".printf((dat.length/dsz)).char_count(); } if (h) { for (int i = 1; i < cw.length; i++) { string hh = "col %u".printf(i); cw[i] = hh.char_count(); } } string o = ""; string ln = ""; // get counts 1st for (int i = 0; i < dat.length; i++) { int c = (i % dsz); if (rnm) { c = c + 1; } int cc = 0; if (dat[i] != null) { cc = dat[i].char_count() + chc + 3; if (cc > 0) { int nwlat = dat[i].index_of("\n"); if (nwlat != -1) { cc = (nwlat + 3 + chc + 3); } } } cw[c] = int.max(cw[c],cc); } // print header if (h) { for (int i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } if (i == 0) { string hh = "#"; o = "%s%s%-*s".printf(o,dlm,cw[i],hh); } else { string hh = "col %d".printf(i-1); o = "%s%s%-*s".printf(o,dlm,cw[i],hh); } if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } // print line for (int i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s|-".printf(tabs); } ln = "%s%s%-*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-|\n".printf(ln); } } o = "%s%s".printf(o,ln); } int n = 0; int rcc = dsz + 1; for (int r = 0; r < rowcount; r++) { for (int c = 0; c < rcc; c++) { string datpart = ""; if (c == 0) { dlm = "%s| ".printf(tabs); o = "%s%s%0*d".printf(o,dlm,cw[0],r); } else { if (c == 1) { dlm = "%s| ".printf(tabs); } int j = (r * dsz) + (c - 1); int cc = dat[j].char_count(); if (dat[j] != null) { if (cc > 0) { datpart = dat[j]; int nwlat = dat[j].index_of("\n"); if (nwlat != -1) { //trunc = true; datpart = (dat[j].substring(0,nwlat) + "..."); cc = (nwlat + 3); } } else { cc = int.max(cc,1); } } dlm = " | "; string clen = ""; if (cnm) { clen = " (%*d)".printf(chc,j); } int padlen = (cw[c] - cc) - clen.char_count(); padlen = int.max(padlen,0); datpart = "%s%s%.*s".printf(datpart,clen,padlen,spc); if (domask) { bool ismasked = (((j % dsz) + m) == 0); if (ismasked) { n += 2; datpart = "\x1b[48;5;5m%s\x1b[0m".printf(datpart); } } o = "%s%s%s".printf(o,dlm,datpart); if (c == dsz) { dlm = " |\n"; o = "%s%s".printf(o,dlm); } } } } cw = null; return o; } string[] readlines (string f) { print("readlines started...\n"); int64 tts = GLib.get_monotonic_time(); string[] o = {}; int maxlines = 0; GLib.File ff = GLib.File.new_for_path(f); if (ff.query_exists()) { GLib.FileStream fstr = GLib.FileStream.open(f,"r"); while (true) { string s = fstr.read_line(); if (s == null) { break; } StringBuilder sbline = new StringBuilder(""); sbline.append(s.strip()); o += strdup(sbline.str); maxlines += 1; } } int64 tte = GLib.get_monotonic_time(); print("read %d lines from %s took %s\n\n".printf(maxlines,f,printusecs(tte-tts))); return o; } string writedat (string p, string n, string x, int dsz, string[] dat) { if ((dat.length == 0) || (dat[0] == null)) { print("no data to write.\n"); return "no data\n"; } string dirname = p; string filename = n; string extname = x; StringBuilder sb = new StringBuilder(""); for (int i = 0; i < dat.length; i++) { if (dat[i] != null) { if (((i % dsz) == (dsz - 1)) && (i > 0)) { sb.append((dat[i] + "\n")); } else { sb.append((dat[i] + ", ")); } } } if ((sb.str.char_count() != 0) && (filename.char_count() != 0) && (extname.char_count() != 0)) { bool allgood = true; string pth = GLib.Environment.get_current_dir(); if (dirname.index_of("/") == 0) { pth = (pth + "/" + dirname + "/"); } else { pth = dirname + "/"; } GLib.Dir dcr = null; try { dcr = Dir.open (pth, 0); } catch (Error e) { print("WRITE checkdir failed: %s\n",e.message); allgood = false; } File hfile = File.new_for_path(pth.concat(filename,".",extname)); File hdir = File.new_for_path(pth); if (allgood == false) { try { hdir.make_directory_with_parents(); allgood = true; } catch (Error e) { print("WRITE makedirs failed: %s\n",e.message); allgood = false; } } if (allgood) { FileOutputStream hose = hfile.replace(null,false,FileCreateFlags.PRIVATE); try { hose.write(sb.str.data); //print("WRITE written to: %s\n",hfile.get_path()); //print("WRITE ended.\n"); return hfile.get_path(); } catch (Error e) { print("WRITE failed: %s\n",e.message); } } else { print("WRITE couldn't make dir, aborting export.\n"); } } else { print("WRITE empty input, aborting.\n"); } return ""; } string[] readdelimited (string f, string rds, string cds, string qc, ref int dsz) { int64 tts = GLib.get_monotonic_time(); string[] dat = {}; GLib.File ff = GLib.File.new_for_path(f); if (ff.query_exists()) { unichar rdc = rds.get_char(0); unichar cdc = cds.get_char(0); char rc = '\x1E'; char cc = '\x1F'; string rcs = "%c".printf(rc); string ccs = "%c".printf(cc); StringBuilder sbfile = new StringBuilder(""); try { uint8[] c; string ex; ff.load_contents (null, out c, out ex); sbfile.append((string) c); } catch (Error e) { print ("\tfailed to read %s: %s\n", ff.get_path(), e.message); } if (qc.char_count() > 0) { string[] quotedsegs = sbfile.str.split(qc); int ofs = 0; if (sbfile.str.index_of_char('\"') == 0) { ofs = 1; } for (int i = ofs; i < quotedsegs.length; i += 2) { StringBuilder sb = new StringBuilder(""); sb.append(quotedsegs[i]); sb.replace(rds,rcs); sb.replace(cds,ccs); quotedsegs[i] = sb.str; } if (ofs == 1) { ofs = 0; } sbfile.erase(0,-1); for (int i = 0; i < quotedsegs.length; i++) { if (((i + ofs) % 2) == 0) { sbfile.append(quotedsegs[i]); } else { sbfile.append(qc + quotedsegs[i] + qc); } } } else { sbfile.replace(rds,rcs); sbfile.replace(cds,ccs); } dsz = 0; string[] rows = sbfile.str.split(rcs); int maxrows = rows.length; //int maxcols = 7; //maxrows = 20; for (int i = 0; i < maxrows; i++) { string[] cols = rows[i].split(ccs); dsz = int.max(dsz,cols.length); //dsz = int.min(dsz,maxcols); } for (int i = 0; i < maxrows; i++) { string[] cols = rows[i].split(ccs); for (int x = 0; x < dsz; x++) { if (x < cols.length) { dat += cols[x].strip(); } else { dat += ""; } } } } int64 tte = GLib.get_monotonic_time(); print("read-delimited from %s took %s\n\n".printf(f,printusecs(tte-tts))); return dat; } public class qixsorter { private int packetnum; private weak int[] dxs; private int dsz; private int fx; private int lx; private int[] bycols; public qixsorter (int pkn, int[] d, int[] bc, int coc, int ff, int ll) { packetnum = pkn; dxs = d; bycols = bc; dsz = coc; fx = ff; lx = ll; } public void quicksortindexbycol (int c, int s, int e, string[] dat) { if ((e - s) > 0) { int p = s; if ((e - s) > 19) { p = (s + ((e - s) / 2)); } int i = s; int j = e; while(i <= j) { while (strintcmp(dat[(dxs[i]*dsz)+c],dat[(dxs[p]*dsz)+c]) < 0) { i += 1; } while (strintcmp(dat[(dxs[j]*dsz)+c],dat[(dxs[p]*dsz)+c]) > 0) { j -= 1; } if (i <= j) { int t = dxs[i]; dxs[i] = dxs[j]; dxs[j] = t; i += 1; j -= 1; } } //if (j < i) { quicksortindexbycol(c,s,j,dat); quicksortindexbycol(c,i,e,dat); //} } } public uint64[] getduperowsbycol(int col, string[] s, int fr, int lr) { // note: start/end range packing sets an upper limit of about 4.2 billion rows // dxs is an array of row numbers, they are scrambled // dsz is the column count // fr is first row in dxs, lr is last row in dxs // to get an absolute index from dxs[i]: (dxs[i] * dsz) + col uint64[] o = {}; uint a = 0; // first row uint b = 0; // last row int r = -1; // recording? for (int i = (fr+1); i <= lr; i++) { int p = (dxs[i] * dsz) + col; int op = (dxs[i-1] * dsz) + col; //print("comparing col %d row %d s[%d] (%s) with col %d row %d s[%d] (%s)\n",col,dxs[i],p,s[p],col,dxs[i-1],op,s[op]); if (strcmp(s[p],s[op]) == 0) { if (r == -1) { // start recording //print("\t1st duplicate row is %d...\n",(i-1)); a = (i-1); o += a; b = 0; r = 1; } else { // keep recording //print("\tlatest duplicate row is %d...\n",(i-1)); b = (i-1); uint64 t = a; o[o.length - 1] = (t << 32) | b; } } else { // stop recording if (r == 1) { //print("\tlast duplicate row is %d...\n",(i-1)); b = (i-1); uint64 t = a; o[o.length - 1] = (t << 32) | b; } a = 0; b = 0; r = -1; } if (i == lr) { // eof if (r == 1) { if (b != lr) { b = lr; uint64 t = a; o[o.length - 1] = (t << 32) | b; } } a = 0; b = 0; r = -1; } } return o; } public void run (ref int packetscomplete, string[] s) { int bc = 0; int a = fx; int b = lx; //print("packet %02d sorting by column %d\n",packetnum,bycols[bc]); quicksortindexbycol(bycols[bc],a,b,s); uint64[] du = getduperowsbycol(bycols[bc],s,a,b); //print("packet %02d initial sort by col %d spawned %d ranges of dupes\n",packetnum,bycols[bc],du.length); int sortnum = 1; while (bc < bycols.length) { //print("\tpacket %2d preparing subsequent sort by col %d with %d dupe ranges\n",packetnum,bycols[bc],du.length); //quicksortindexbycol(bycols[bc],a,b,s); if (du.length > 0) { bc += 1; if (bc < bycols.length) { uint64[] xd = {}; for (int q = 0; q < du.length; q++) { uint64 bb = (du[q] & 0xFFFFFFFF); uint64 aa = (du[q] >> 32); if (bb > aa) { a = ((int) aa); b = ((int) bb); //print("\tpacket %02d re-sorting by col %d from packet row %d to %d of %d\n",packetnum,bycols[bc],a,b,(dxs.length - 1)); quicksortindexbycol(bycols[bc],a,b,s); uint64[] dd = getduperowsbycol(bycols[bc],s,a,b); //print("\tpacket %02d subsequent sort by col %d spawned %d ranges of dupes\n",packetnum,bycols[bc],dd.length); for (int t = 0; t < dd.length; t++) { xd += dd[t]; }; //print("\tpacket %02d subsequent sort by col %d accumulating %d ranges of dupes\n",packetnum,bycols[bc],xd.length); sortnum += 1; } } du.resize(0); for (int t = 0; t < xd.length; t++) { du += xd[t]; }; } else { break; } } else { break; } // nothing else to sort //bc += 1; } bc = int.min(bc,(bycols.length - 1)); //print("\t\tpacket %d (%d..%d) sort %d was by col %d (%d/%d), which produced %d dupes\n",packetnum,fx,lx,sortnum,bycols[bc],bc,(bycols.length - 1),du.length); mutex.lock(); packetscomplete += 1; mutex.unlock(); } } public class datcopybyindex { private int packetnum; private int[] dxs; private int aa; public datcopybyindex (int pkn, int ofs, int[] dx) { packetnum = pkn; dxs = dx; aa = ofs; } public void run (ref int packetscomplete, ref string[] o, string[] dat, int colcount) { //print("\t\tpacket %02d copying to range from o[%05d] to o[%d]...\n",packetnum,(aa*colcount),(((aa+(dxs.length - 1))*colcount)+(colcount - 1))); for (int j = 0; j < dxs.length; j++) { int a = dxs[j] * colcount; int b = a + colcount; int n = 0; for (int g = a; g < b; g++) { o[((aa+j) * colcount)+n] = strdup(dat[g]); n += 1; } } mutex.lock(); packetscomplete += 1; mutex.unlock(); } } int[] datgetcol (int p, int dsz, int l) { int[] o = {}; for (int i = 0; i < l; i++) { if (((i - p) % dsz) == 0) { o += i; } } return o; } void datinsertcol (ref string[] dat, int p, int dsz, int[] cx, string[] bdat) { int a = dsz + 1; int j = (int) (dat.length / dsz); int n = j - 1; dat.resize(dat.length + j); for (int i = (dat.length - 1); i >= 0; i--) { int r = (int) (i / dsz); if (((i-p) % a) == 0) { dat[i] = bdat[cx[n]]; n -= 1; j -= 1; } else { dat[i] = dat[i-j]; } } } void main() { mutex = GLib.Mutex(); int nthr = int.max(1,((int) GLib.get_num_processors())); GLib.Intl.setlocale(ALL,""); // {'-','+','0','1','2','3','4','5','6','7','8','9','.'}; numchars = {43,45,48,49,50,51,52,53,54,55,56,57,46}; // TAB SPC " $ ' prenums = {9, 32, 34, 36, 39}; // ascii partitions int rng = 127; int n = 0; int[] chrs = {}; for (int i = 0; i < rng; i++) { chrs += i; } //print("first is %d\n",0); //print("last is 126\n"); //print("length is %d\n",rng); print("\ncheck GLib2 sorting, is the ascii table the same as the sort order?\n"); string[] abc = {}; string[] cba = {}; for (int i = 32; i < 127; i++) { abc += "%d".printf(i); abc += ((char) i).to_string(); } for (int i = 0; i < abc.length; i++) { cba += abc[i]; } quicksortstringsbycol(ref abc, 3, 1, 2, 0, ((abc.length / 2) - 1)); int[] ncol = datgetcol(0,2,abc.length); datinsertcol(ref cba,-1,2,ncol,abc); ncol = datgetcol(1,2,abc.length); datinsertcol(ref cba,-1,3,ncol,abc); print ("\nunsorted and sorted ascii chars:\n%s\n",printunrolledarray(1,cba,4,true,true,false,-1)); //print("\"[\" offset from \"<\" is %d\n\n",strcmp("[","<")); // ordered lists to pick from string[] letters = {"AA","BB","CC","DD","EE","FF"}; string[] nums = {"-1","1,1", "2.1", "\"$11,000.00\"","\"$30,000.00\"","2/2/22"}; string[] misc = {"!!","$$","..","<<","@@","[["}; int datsize = 1; int[] bycols = {6,2,74}; int colcount = 3; int col = 1; string[] s = {}; int rando = Random.int_range(0,5); int rs = 0; for (int i = 0; i < 30 ; i++) { s += letters[rando]; rs += 1; Random.set_seed(rs); rando = Random.int_range(0,6); s += nums[rando]; rs += 1; Random.set_seed(rs); rando = Random.int_range(0,6); s += misc[rando]; rs += 1; Random.set_seed(rs); rando = Random.int_range(0,6); } if (datsize == 0) { col = 0; bycols = {0,1,2}; } if (datsize == 1) { //s = readlines("./cpbrown_notes.org"); s = readlines("./leipzig1M.txt"); colcount = 1; col = 0; bycols = {0}; } if (datsize == 2) { s = readdelimited("./LCA_Disclosure_Data_FY2022_Q1.csv","\n",",","\"",ref colcount); col = 6; bycols = {6,2,74}; } print("ascii partitioning started...\n\n"); print("\tmaking ascii partitions...\n"); int64 ptts = GLib.get_monotonic_time(); int64 tts = GLib.get_monotonic_time(); int[] asciiscores = {}; for (int i = 0; i < rng; i++) { asciiscores += i; asciiscores += 0; asciiscores += 0; } for (int i = col; i < s.length; i += colcount) { unichar ff = s[i].get_char(0); int fx = ((int) ff); if (fx < 127) { // file char count under its char index at col 1 of 2 asciiscores[(fx*3)+2] += 1; asciiscores[(fx*3)+1] = fx; } else { // char is out of range, file it under 1st item c[0] asciiscores[2] += 1; asciiscores[1] = -1; } } int nn = 0; for (int i = 0; i < asciiscores.length; i += 3) { if (asciiscores[i+2] > 0) { asciiscores[i] = nn; nn += 1; } } int[] charids = {}; double[] charcountof = {}; int[] charidxs = {}; double[] charcounts = {}; int mcc = 0; int tcc = 0; for (int i = 0; i < (asciiscores.length - 3); i += 3) { if (asciiscores[i+2] > 0) { charids += asciiscores[i+1]; mcc = int.max(mcc,asciiscores[i+2]); } } for (int i = 0; i < (asciiscores.length - 3); i += 3) { if (asciiscores[i+2] > 0) { charidxs += charids[asciiscores[i]]; charcounts += (asciiscores[i+2] * 1.0); tcc += asciiscores[i+2]; charcountof += ((asciiscores[i+2] * 1.0) / mcc); char an = ((char) asciiscores[i+1]); //print("%03d | %03d (%s) | %03d\n",asciiscores[i],asciiscores[i+1],an.to_string(),asciiscores[i+2]); } } int icc = tcc / nthr; //print("balanced item count per paket is %d\n",icc); //print("total item count is %d\n",tcc); //print("charids.length is %d\n",charids.length); double[] dnup = new double[0]; packet[] qsegs = packseq(0,charids,charids.length,true,nthr,1,0,"FIT",charcounts,tcc); for (int i = 0; i < qsegs.length; i++) { int qcc = qsegs[i].items.length; //for (int g = 0; g < qsegs[i].items.length; g++) { // print("qsegs[%d].items[%d] is %d\n",i,g,qsegs[i].items[g]); //} //print("qseg[%d] has a total item count of %d\n",i,qcc); } int64 tte = GLib.get_monotonic_time(); print("\tascii partitioning took %s\n\n".printf(printusecs(tte-tts))); // index rows by packet print("\tfiling row/line into partitions...\n"); tts = GLib.get_monotonic_time(); packet[] bins = binstrings(s,qsegs,col,colcount); tte = GLib.get_monotonic_time(); print("\tfiling took %s\n\n".printf(printusecs(tte-tts))); int64 ptte = GLib.get_monotonic_time(); print("all ascii partitioning took %s\n\n".printf(printusecs(ptte-ptts))); // qsort rows in each packet, if required print("single threaded sort...\n\n"); print("\tsorting indices...\n"); int64 atts = GLib.get_monotonic_time(); tts = GLib.get_monotonic_time(); int nop = 0; for (int i = 0; i < bins.length; i++) { if (bins[i].items.length > 1) { qixsorter qsx = new qixsorter(i,bins[i].items,bycols,colcount,0,(bins[i].items.length - 1)); qsx.run(ref nop, s); //quicksortindexbycol(ref bins[i].items, col, colcount, 0, (bins[i].items.length - 1), s); } } tte = GLib.get_monotonic_time(); print("\tsorting took %s\n\n".printf(printusecs(tte-tts))); // output using packets print("\tcopying to output...\n"); tts = GLib.get_monotonic_time(); string[] ous = {}; for (int i = 0; i < bins.length; i++) { for (int j = 0; j < bins[i].items.length; j++) { int a = bins[i].items[j] * colcount; int b = a + colcount; for (int g = a; g < b; g++) { ous += strdup(s[g]); } } } tte = GLib.get_monotonic_time(); print("\tcopying to output took %s\n\n".printf(printusecs(tte-tts))); int64 atte = GLib.get_monotonic_time(); print("all single-threaded sort operations took %s\n\n".printf(printusecs((atte-atts)+(ptte-ptts)))); // multithreaded test if (datsize > 0) { print("multithreaded sort...\n\n"); print("\tsorting indices...\n"); int64 mtts = GLib.get_monotonic_time(); tts = GLib.get_monotonic_time(); int rowcount = s.length / colcount; int packetscomplete = 0; ThreadPool<qixsorter> tptf = new ThreadPool<qixsorter>.with_owned_data( (qixsorter) => { qixsorter.run( ref packetscomplete, s ); }, nthr, false ); for (int i = 0; i < bins.length; i++) { tptf.add ( new qixsorter( i, bins[i].items, bycols, colcount, 0, (bins[i].items.length - 1) ) ); //print("\tprocessing packet %02d of %d\n",i,(bins.length - 1)); } while (packetscomplete != bins.length) { GLib.Thread.usleep(1); } tte = GLib.get_monotonic_time(); print("\tmultithreaded sort took %s\n\n".printf(printusecs(tte-tts))); print("\tresetting output array...\n"); tts = GLib.get_monotonic_time(); string[] oo = {}; oo.resize(s.length); tte = GLib.get_monotonic_time(); print("\tresetting took %s\n\n".printf(printusecs(tte-tts))); print("\tmultithreaded copy...\n"); tts = GLib.get_monotonic_time(); packetscomplete = 0; ThreadPool<datcopybyindex> tpdc = new ThreadPool<datcopybyindex>.with_owned_data( (datcopybyindex) => { datcopybyindex.run( ref packetscomplete, ref oo, s, colcount ); }, nthr, false ); int ofs = 0; for (int i = 0; i < bins.length; i++) { tpdc.add ( new datcopybyindex( i, ofs, bins[i].items ) ); ofs += bins[i].items.length; //print("\tprocessing packet %02d of %d\n",i,(bins.length - 1)); } while (packetscomplete != bins.length) { GLib.Thread.usleep(1); } tte = GLib.get_monotonic_time(); print("\tmultithreaded copy took %s\n\n".printf(printusecs(tte-tts))); int64 mtte = GLib.get_monotonic_time(); print("all multithreaded sort operations took %s\n\n".printf(printusecs((mtte-mtts)+(ptte-ptts)))); print("saving output to file...\n"); if (datsize == 1) { string oup = writedat("./","asciisort_test","txt",colcount,oo); } else { string oup = writedat("./","asciisort_test","csv",colcount,oo); } } if (datsize == 0) { print("source data:\n%s\n",printunrolledarray(1,s,colcount,true,true,false,-1)); print("output data:\n%s\n",printunrolledarray(1,ous,colcount,true,true,false,-1)); } }
#+RESULTS:
Compilation succeeded - 27 warning(s) check GLib2 sorting, is the ascii table the same as the sort order? unsorted and sorted ascii chars: | # | col 0 | col 1 | col 2 | col 3 | | | |----+-------+-------+-------+-------+---+---| | 00 | 32 | | 32 | | | | | 01 | 33 | ! | 33 | ! | | | | 02 | 34 | " | 34 | " | | | | 03 | 35 | # | 35 | # | | | | 04 | 36 | $ | 36 | $ | | | | 05 | 37 | % | 37 | % | | | | 06 | 38 | & | 38 | & | | | | 07 | 39 | ' | 39 | ' | | | | 08 | 40 | ( | 40 | ( | | | | 09 | 41 | ) | 41 | ) | | | | 10 | 42 | * | 42 | * | | | | 11 | 43 | + | 43 | + | | | | 12 | 44 | , | 44 | , | | | | 13 | 45 | - | 45 | - | | | | 14 | 46 | . | 46 | . | | | | 15 | 47 | / | 47 | / | | | | 16 | 48 | 0 | 48 | 0 | | | | 17 | 49 | 1 | 49 | 1 | | | | 18 | 50 | 2 | 50 | 2 | | | | 19 | 51 | 3 | 51 | 3 | | | | 20 | 52 | 4 | 52 | 4 | | | | 21 | 53 | 5 | 53 | 5 | | | | 22 | 54 | 6 | 54 | 6 | | | | 23 | 55 | 7 | 55 | 7 | | | | 24 | 56 | 8 | 56 | 8 | | | | 25 | 57 | 9 | 57 | 9 | | | | 26 | 58 | : | 58 | : | | | | 27 | 59 | ; | 59 | ; | | | | 28 | 60 | < | 60 | < | | | | 29 | 61 | = | 61 | = | | | | 30 | 62 | > | 62 | > | | | | 31 | 63 | ? | 63 | ? | | | | 32 | 64 | @ | 64 | @ | | | | 33 | 65 | A | 65 | A | | | | 34 | 66 | B | 66 | B | | | | 35 | 67 | C | 67 | C | | | | 36 | 68 | D | 68 | D | | | | 37 | 69 | E | 69 | E | | | | 38 | 70 | F | 70 | F | | | | 39 | 71 | G | 71 | G | | | | 40 | 72 | H | 72 | H | | | | 41 | 73 | I | 73 | I | | | | 42 | 74 | J | 74 | J | | | | 43 | 75 | K | 75 | K | | | | 44 | 76 | L | 76 | L | | | | 45 | 77 | M | 77 | M | | | | 46 | 78 | N | 78 | N | | | | 47 | 79 | O | 79 | O | | | | 48 | 80 | P | 80 | P | | | | 49 | 81 | Q | 81 | Q | | | | 50 | 82 | R | 82 | R | | | | 51 | 83 | S | 83 | S | | | | 52 | 84 | T | 84 | T | | | | 53 | 85 | U | 85 | U | | | | 54 | 86 | V | 86 | V | | | | 55 | 87 | W | 87 | W | | | | 56 | 88 | X | 88 | X | | | | 57 | 89 | Y | 89 | Y | | | | 58 | 90 | Z | 90 | Z | | | | 59 | 91 | [ | 91 | [ | | | | 60 | 92 | \ | 92 | \ | | | | 61 | 93 | ] | 93 | ] | | | | 62 | 94 | ^ | 94 | ^ | | | | 63 | 95 | _ | 95 | _ | | | | 64 | 96 | ` | 96 | ` | | | | 65 | 97 | a | 97 | a | | | | 66 | 98 | b | 98 | b | | | | 67 | 99 | c | 99 | c | | | | 68 | 100 | d | 100 | d | | | | 69 | 101 | e | 101 | e | | | | 70 | 102 | f | 102 | f | | | | 71 | 103 | g | 103 | g | | | | 72 | 104 | h | 104 | h | | | | 73 | 105 | i | 105 | i | | | | 74 | 106 | j | 106 | j | | | | 75 | 107 | k | 107 | k | | | | 76 | 108 | l | 108 | l | | | | 77 | 109 | m | 109 | m | | | | 78 | 110 | n | 110 | n | | | | 79 | 111 | o | 111 | o | | | | 80 | 112 | p | 112 | p | | | | 81 | 113 | q | 113 | q | | | | 82 | 114 | r | 114 | r | | | | 83 | 115 | s | 115 | s | | | | 84 | 116 | t | 116 | t | | | | 85 | 117 | u | 117 | u | | | | 86 | 118 | v | 118 | v | | | | 87 | 119 | w | 119 | w | | | | 88 | 120 | x | 120 | x | | | | 89 | 121 | y | 121 | y | | | | 90 | 122 | z | 122 | z | | | | 91 | 123 | { | 123 | { | | | | 92 | 124 | | | 124 | | | | 93 | 125 | } | 125 | } | | | | 94 | 126 | ~ | 126 | ~ | | | readlines started... read 1000000 lines from ./leipzig1M.txt took 1.20 s ascii partitioning started... making ascii partitions... ascii partitioning took 41.14 ms filing row/line into partitions... filing took 355.65 ms all ascii partitioning took 396.82 ms single threaded sort... sorting indices... sorting took 3.01 s copying to output... copying to output took 287.12 ms all single-threaded sort operations took 3.69 s multithreaded sort... sorting indices... multithreaded sort took 966.23 ms resetting output array... resetting took 3.33 ms multithreaded copy... multithreaded copy took 95.71 ms all multithreaded sort operations took 1.46 s saving output to file...
BIASGAIN:vala:

Biasgain test
Based on the work of Christophe Schlick : https:dept-info.labri.fr/~schlick/DOC/gem2.html

// biasgain test // by c.p.brown 2025 // this is derivative work from various sources, // most of which are based on Graphics Gems IV, p379-382, April 1994, by Christophe Schlick // https://dept-info.labri.fr/~schlick/DOC/gem2.html // double biasgain (double x, double s, double f) { double g = x; if ((x < 0.5) || (f == 1.0)) { g = ((x * f) / ((((1.0 / s) - 2.0) * (1.0 - (f * x))) + 1.0)) / f; } else { g = ((((x * f) - 1.0) / ((((1.0 / (1.0 - s)) - 2.0) * (1.0 - ( (f * x) - 1.0))) + 1.0)) / f) + 0.5; } return g; } void main() { GLib.Intl.setlocale(ALL,""); int st = 0; print("s = 0.25\n"); for (int n = 0; n < 20; n++) { double x = (n * 1.0) / 19.0; double s = 0.25; double f = 2.0; double g = biasgain(x,s,f); int w = 30; int bar = ((int) (g * w)); string plot = ""; for (int j = 0; j < bar; j++) { plot = plot + "█"; } print("%02d | %0.2f | %0.2f\t| %02d | %s\n",n,x,g,bar,plot); } print("\n"); print("s = 0.5\n"); for (int n = 0; n < 20; n++) { double x = (n * 1.0) / 19.0; double s = 0.5; double f = 2.0; double g = biasgain(x,s,f); int w = 30; int bar = ((int) (g * w)); string plot = ""; for (int j = 0; j < bar; j++) { plot = plot + "█"; } print("%02d | %0.2f | %0.2f\t| %02d | %s\n",n,x,g,bar,plot); } print("\n"); print("s = 0.75\n"); for (int n = 0; n < 20; n++) { double x = (n * 1.0) / 19.0; double s = 0.75; double f = 2.0; double g = biasgain(x,s,f); int w = 30; int bar = ((int) (g * w)); string plot = ""; for (int j = 0; j < bar; j++) { plot = plot + "█"; } print("%02d | %0.2f | %0.2f\t| %02d | %s\n",n,x,g,bar,plot); } print("\n"); print("s = 0.75, f = 1.0\n"); for (int n = 0; n < 20; n++) { double x = (n * 1.0) / 19.0; double s = 0.75; double f = 1.0; double g = biasgain(x,s,f); int w = 30; int bar = ((int) (g * w)); string plot = ""; for (int j = 0; j < bar; j++) { plot = plot + "█"; } print("%02d | %0.2f | %0.2f\t| %02d | %s\n",n,x,g,bar,plot); } print("\n"); print("s = 0.25, f = 1.0\n"); for (int n = 0; n < 20; n++) { double x = (n * 1.0) / 19.0; double s = 0.25; double f = 1.0; double g = biasgain(x,s,f); int w = 30; int bar = ((int) (g * w)); string plot = ""; for (int j = 0; j < bar; j++) { plot = plot + "█"; } print("%02d | %0.2f | %0.2f\t| %02d | %s\n",n,x,g,bar,plot); } print("\n"); }
#+RESULTS:
Compilation succeeded - 1 warning(s) s = 0.25 00 | 0.00 | 0.00 | 00 | 01 | 0.05 | 0.02 | 00 | 02 | 0.11 | 0.04 | 01 | █ 03 | 0.16 | 0.07 | 02 | ██ 04 | 0.21 | 0.10 | 02 | ██ 05 | 0.26 | 0.14 | 04 | ████ 06 | 0.32 | 0.18 | 05 | █████ 07 | 0.37 | 0.24 | 07 | ███████ 08 | 0.42 | 0.32 | 09 | █████████ 09 | 0.47 | 0.43 | 12 | ████████████ 10 | 0.53 | 0.57 | 17 | █████████████████ 11 | 0.58 | 0.68 | 20 | ████████████████████ 12 | 0.63 | 0.76 | 22 | ██████████████████████ 13 | 0.68 | 0.82 | 24 | ████████████████████████ 14 | 0.74 | 0.86 | 25 | █████████████████████████ 15 | 0.79 | 0.90 | 27 | ███████████████████████████ 16 | 0.84 | 0.93 | 28 | ████████████████████████████ 17 | 0.89 | 0.96 | 28 | ████████████████████████████ 18 | 0.95 | 0.98 | 29 | █████████████████████████████ 19 | 1.00 | 1.00 | 30 | ██████████████████████████████ s = 0.5 00 | 0.00 | 0.00 | 00 | 01 | 0.05 | 0.05 | 01 | █ 02 | 0.11 | 0.11 | 03 | ███ 03 | 0.16 | 0.16 | 04 | ████ 04 | 0.21 | 0.21 | 06 | ██████ 05 | 0.26 | 0.26 | 07 | ███████ 06 | 0.32 | 0.32 | 09 | █████████ 07 | 0.37 | 0.37 | 11 | ███████████ 08 | 0.42 | 0.42 | 12 | ████████████ 09 | 0.47 | 0.47 | 14 | ██████████████ 10 | 0.53 | 0.53 | 15 | ███████████████ 11 | 0.58 | 0.58 | 17 | █████████████████ 12 | 0.63 | 0.63 | 18 | ██████████████████ 13 | 0.68 | 0.68 | 20 | ████████████████████ 14 | 0.74 | 0.74 | 22 | ██████████████████████ 15 | 0.79 | 0.79 | 23 | ███████████████████████ 16 | 0.84 | 0.84 | 25 | █████████████████████████ 17 | 0.89 | 0.89 | 26 | ██████████████████████████ 18 | 0.95 | 0.95 | 28 | ████████████████████████████ 19 | 1.00 | 1.00 | 30 | ██████████████████████████████ s = 0.75 00 | 0.00 | 0.00 | 00 | 01 | 0.05 | 0.13 | 03 | ███ 02 | 0.11 | 0.22 | 06 | ██████ 03 | 0.16 | 0.29 | 08 | ████████ 04 | 0.21 | 0.34 | 10 | ██████████ 05 | 0.26 | 0.38 | 11 | ███████████ 06 | 0.32 | 0.42 | 12 | ████████████ 07 | 0.37 | 0.45 | 13 | █████████████ 08 | 0.42 | 0.47 | 14 | ██████████████ 09 | 0.47 | 0.49 | 14 | ██████████████ 10 | 0.53 | 0.51 | 15 | ███████████████ 11 | 0.58 | 0.53 | 15 | ███████████████ 12 | 0.63 | 0.55 | 16 | ████████████████ 13 | 0.68 | 0.58 | 17 | █████████████████ 14 | 0.74 | 0.62 | 18 | ██████████████████ 15 | 0.79 | 0.66 | 19 | ███████████████████ 16 | 0.84 | 0.71 | 21 | █████████████████████ 17 | 0.89 | 0.78 | 23 | ███████████████████████ 18 | 0.95 | 0.87 | 26 | ██████████████████████████ 19 | 1.00 | 1.00 | 30 | ██████████████████████████████ s = 0.75, f = 1.0 00 | 0.00 | 0.00 | 00 | 01 | 0.05 | 0.14 | 04 | ████ 02 | 0.11 | 0.26 | 07 | ███████ 03 | 0.16 | 0.36 | 10 | ██████████ 04 | 0.21 | 0.44 | 13 | █████████████ 05 | 0.26 | 0.52 | 15 | ███████████████ 06 | 0.32 | 0.58 | 17 | █████████████████ 07 | 0.37 | 0.64 | 19 | ███████████████████ 08 | 0.42 | 0.69 | 20 | ████████████████████ 09 | 0.47 | 0.73 | 21 | █████████████████████ 10 | 0.53 | 0.77 | 23 | ███████████████████████ 11 | 0.58 | 0.80 | 24 | ████████████████████████ 12 | 0.63 | 0.84 | 25 | █████████████████████████ 13 | 0.68 | 0.87 | 26 | ██████████████████████████ 14 | 0.74 | 0.89 | 26 | ██████████████████████████ 15 | 0.79 | 0.92 | 27 | ███████████████████████████ 16 | 0.84 | 0.94 | 28 | ████████████████████████████ 17 | 0.89 | 0.96 | 28 | ████████████████████████████ 18 | 0.95 | 0.98 | 29 | █████████████████████████████ 19 | 1.00 | 1.00 | 30 | ██████████████████████████████ s = 0.25, f = 1.0 00 | 0.00 | 0.00 | 00 | 01 | 0.05 | 0.02 | 00 | 02 | 0.11 | 0.04 | 01 | █ 03 | 0.16 | 0.06 | 01 | █ 04 | 0.21 | 0.08 | 02 | ██ 05 | 0.26 | 0.11 | 03 | ███ 06 | 0.32 | 0.13 | 04 | ████ 07 | 0.37 | 0.16 | 04 | ████ 08 | 0.42 | 0.20 | 05 | █████ 09 | 0.47 | 0.23 | 06 | ██████ 10 | 0.53 | 0.27 | 08 | ████████ 11 | 0.58 | 0.31 | 09 | █████████ 12 | 0.63 | 0.36 | 10 | ██████████ 13 | 0.68 | 0.42 | 12 | ████████████ 14 | 0.74 | 0.48 | 14 | ██████████████ 15 | 0.79 | 0.56 | 16 | ████████████████ 16 | 0.84 | 0.64 | 19 | ███████████████████ 17 | 0.89 | 0.74 | 22 | ██████████████████████ 18 | 0.95 | 0.86 | 25 | █████████████████████████ 19 | 1.00 | 1.00 | 30 | ██████████████████████████████
BITWISE:vala:

Misc bitwise tests.

string printallmyuint8bits (uint8 n) { int b = 7; string o = ""; for (int i = b; i >= 0; i--) { o = o + "%d".printf((n >> i) & 1); } return o; } void main() { print("\nstoring id + value...\n\n"); uint i = 123456789; print("index i is %u, value is 1\n",i); print("(i |= 1) is %u\n",(i |= 1)); print("(i & 1) is %u\n",(i & 1)); print("i & ~(1 << 1) is %u\n",(i & ~(1 << 1))); i = 123456789; print("\nindex i is %u, value is 0\n",i); print("(i |= 0) is %u\n",(i |= 0)); print("(i & 1) is %u\n",(i & 0)); print("i & ~(1 << 1) is %u\n",(i & ~(1 << 1))); i = 123456789; print("\nindex i is %u, value is 1\n",i); i = ((i << 1) | 1); print("(i << 1) |= 1 is %u\n",i); print("(i & 1) is %u\n",(i & 1)); print("(i >> 1) is %u\n",(i >> 1)); i = 123456789; print("\nindex i is %u, value is 0\n",i); i = ((i << 1) | 0); print("(i << 1) |= 0 is %u\n",i); print("(i & 1) is %u\n",(i & 1)); print("(i >> 1) is %u\n",(i >> 1)); i = 123456789; uint v = 3; print("\nindex i is %u, value v is %u\n",i,v); uint r = ((i << 4) | v); print("(i << 4) | %u is %u\n",v,r); print("(i & 0xF) is %u\n",(r & 0xF)); print("(i >> 4) is %u\n",(r >> 4)); i = 123456789; v = 15; print("\nindex i is %u, value v is %u (max @ 4bit)\n",i,v); r = ((i << 4) | v); print("(i << 4) | %u is %u\n",v,r); print("(i & 0xF) is %u\n",(r & 0xF)); print("(i >> 4) is %u\n",(r >> 4)); i = 123456789; v = 255; print("\nindex i is %u, value v is %u (max @ 8bit)\n",i,v); r = ((i << 8) | v); print("(i << 8) | %u is %u\n",v,r); print("(i & 0xFF) is %u\n",(r & 0xFF)); print("(i >> 8) is %u -- not enough free bits\n",(r >> 8)); i = 12; v = 255; print("\nindex i is %u, value v is %u (max @ 8bit)\n",i,v); r = ((i << 8) | v); print("(i << 8) | %u is %u\n",v,r); print("(i & 0xFF) is %u\n",(r & 0xFF)); print("(i >> 8) is %u -- works as %u has enough bits to spare\n",(r >> 8),i); i = 1000000000; print("\nreal-world case, store a 1 or 0 value with a cell index in a large-ish array:\nindex i is %u, value is 1\n",i); i = ((i << 1) | 1); print("(i << 1) |= 1 is %u\n",i); print("(i & 1) is %u\n",(i & 1)); print("(i >> 1) is %u\n",(i >> 1)); i = 1000000000; print("\nindex i is %u, value v is 0\n",i); i = ((i << 1) | 0); print("(i << 1) |= 0 is %u\n",i); print("(i & 1) is %u\n",(i & 1)); print("(i >> 1) is %u\n",(i >> 1)); uint64 q = 1000000000; print("\nindex q is %lld, value v is 500 000 000\n",q); q = ((q << 30) | 500000000); print("(q << 30) | 500000000 is %lld\n",q); print("(q & 30) is %lld\n",(q & 30)); print("(q & ((1 << 30) - 1)) is %lld\n",(q & ((1 << 30) - 1))); print("(q >> 30) is %lld\n",(q >> 30)); q = 1000000000; print("\nindex q is %lld, value v is 500 000 000\n",q); q = ((q << 32) | 500000000); print("(q << 32) | 500000000 is %lld\n",q); print("(q & 32) is %lld\n",(q & 32)); print("(q & 0xFFFFFFFF) is %lld\n",(q & 0xFFFFFFFF)); print("(q >> 32) is %lld\n",(q >> 32)); q = 1000000000; print("\nindex q is %lld, value v is 500 000 000\n",q); q = ((q << 32) | 500000000); print("(q << 32) | 500000000 is %lld\n",q); print("(q & 32) is %lld\n",(q & 32)); print("(q >> 32) is %lld\n",(q >> 32)); print("(q & 0xFFFFFFFF) is %lld\n",(q & 0xFFFFFFFF)); uint8 j = 10; uint8 n = 13; print("\nindex j is %d (%s), value is %d (%s)\n",j,printallmyuint8bits(j),n,printallmyuint8bits(n)); j = ((j << 4) | n); print("(j << 4) | 13 is %d (%s)\n",j,printallmyuint8bits(j)); print("(j & 0xF) is %d (%s)\n",(j & 0xF),printallmyuint8bits(j & 0xF)); j = (j >> 4); print("(j >> 4) is %d (%s)\n",j,printallmyuint8bits(j)); int ll = 91; int rr = 95; print("\nll is %d, rr is %d\n",ll,rr); print("(ll+rr)>>1) is %d\n",((ll+rr)>>1)); print("((rr-ll)/2)+ll is %d\n\n",(((rr-ll)/2)+ll)); int ee = 22; int oo = 33; print("(%d & 1) is %d\n",ee,(ee & 1)); print("(%d & 1) is %d\n\n",oo,(oo & 1)); }
#+RESULTS:
storing id + value... index i is 123456789, value is 1 (i |= 1) is 123456789 (i & 1) is 1 i & ~(1 << 1) is 123456789 index i is 123456789, value is 0 (i |= 0) is 123456789 (i & 1) is 0 i & ~(1 << 1) is 123456789 index i is 123456789, value is 1 (i << 1) |= 1 is 246913579 (i & 1) is 1 (i >> 1) is 123456789 index i is 123456789, value is 0 (i << 1) |= 0 is 246913578 (i & 1) is 0 (i >> 1) is 123456789 index i is 123456789, value v is 3 (i << 4) | 3 is 1975308627 (i & 0xF) is 3 (i >> 4) is 123456789 index i is 123456789, value v is 15 (max @ 4bit) (i << 4) | 15 is 1975308639 (i & 0xF) is 15 (i >> 4) is 123456789 index i is 123456789, value v is 255 (max @ 8bit) (i << 8) | 255 is 1540167167 (i & 0xFF) is 255 (i >> 8) is 6016277 -- not enough free bits index i is 12, value v is 255 (max @ 8bit) (i << 8) | 255 is 3327 (i & 0xFF) is 255 (i >> 8) is 12 -- works as 12 has enough bits to spare real-world case, store a 1 or 0 value with a cell index in a large-ish array: index i is 1000000000, value is 1 (i << 1) |= 1 is 2000000001 (i & 1) is 1 (i >> 1) is 1000000000 index i is 1000000000, value v is 0 (i << 1) |= 0 is 2000000000 (i & 1) is 0 (i >> 1) is 1000000000 index q is 1000000000, value v is 500 000 000 (q << 30) | 500000000 is 1073741824500000000 (q & 30) is 0 (q & ((1 << 30) - 1)) is 500000000 (q >> 30) is 1000000000 index q is 1000000000, value v is 500 000 000 (q << 32) | 500000000 is 4294967296500000000 (q & 32) is 0 (q & 0xFFFFFFFF) is 500000000 (q >> 32) is 1000000000 index q is 1000000000, value v is 500 000 000 (q << 32) | 500000000 is 4294967296500000000 (q & 32) is 0 (q >> 32) is 1000000000 (q & 0xFFFFFFFF) is 500000000 index j is 10 (00001010), value is 13 (00001101) (j << 4) | 13 is 173 (10101101) (j & 0xF) is 13 (00001101) (j >> 4) is 10 (00001010) ll is 91, rr is 95 (ll+rr)>>1) is 93 ((rr-ll)/2)+ll is 93 (22 & 1) is 0 (33 & 1) is 1
CASE:vala:

String cleanup for searching.
string.down() & string.up() return copies of string.
In-place _.strip() spews a bunch of warnings, but the resulting string is valid.

void main() { string s = " AAbbCC "; if (s.validate()) { s.down(); s.strip(); print("\"%s\"\n",s); } else { print("string is not null-terminated\n"); } s = " AAbbCC "; s._strip(); if (s.validate()) { print("\"%s\"\n",s); } else { print("string after _strip() is not null-terminated\n"); } s = " AAbbCC "; s = s.down().strip(); if (s.validate()) { print("\"%s\"\n",s); } else { print("string after .down().strip() is not null-terminated\n"); } }
#+RESULTS:
" AAbbCC " "AAbbCC" "aabbcc"
CHARCOUNT:vala:
// get number of char in string // by c.p.brown 2024 void main() { string s = "\"\na sentence\nwith some\nnewlines\nin it\n\""; int n = 0; // index to search from int nn = 0; // found index int cc = -1; // found char count while (nn != -1) { cc += 1; nn = s.index_of_char('\n',n); n = (nn + 1); } print("there are %d newlines in this string:\n%s",cc,s); string sorg = ""; string testfile = "cpbrown_notes.org"; string ff = Path.build_filename ("./", testfile); GLib.File og = File.new_for_path(ff); if (og.query_exists() == true) { try { uint8[] c; string e; og.load_contents (null, out c, out e); sorg = (string) c; } catch (Error e) { print ("\tfailed to read %s: %s\n", og.get_path(), e.message); } } int64 tts = GLib.get_real_time(); n = 0; nn = 0; cc = -1; while (nn != -1) { cc += 1; nn = sorg.index_of_char('\n',n); n = (nn + 1); } print("\n\nthere are %d newlines in this orgfile\n",cc); int64 tte = GLib.get_real_time(); print("it took %.6f seconds to count them all\n",((double) (tte - tts) / 1000000.0)); }
#+RESULTS:
there are 5 newlines in this string: " a sentence with some newlines in it " there are 11063 newlines in this orgfile it took 0.000151 seconds to count them all
COLCTRL:vala:
// getting & setting columns // oop dresscode is for easier integration with another project // by c.p.brown 2024 void snortint (int[] l) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = l.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; j -= g; } l[j] = t; } } } int[] snortintindi (int[] l) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = l.length; int[] o = new int[n]; for (int i = 0; i < n; i++) { o[i] = i; } foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int q = o[i]; int j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; o[j] = o[j - g]; j -= g; } l[j] = t; o[j] = q; } } return o; } public class MyDat : Object { public int cols; public int[] dat; public int[] copyrow (int p) { int64 tts = GLib.get_monotonic_time(); int[] o = new int[cols]; int a = p * cols; int b = a + cols; for (int i = a; i < b; i++) { o[i - a] = dat[i]; } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t copyrow took %.2f microseconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t copyrow took %.2f microseconds\n",((double) (tte - tts))); } return o; } public string printrow (int p) { int64 tts = GLib.get_monotonic_time(); string o = "{"; int a = p * cols; int b = a + cols; for (int i = a; i < (b - 1); i++) { o = (o + "%d".printf(dat[i]) + ","); } o = (o + "%d".printf(dat[b - 1]) + "}"); int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t printrow took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t printrow took %.2f microseconds\n",((double) (tte - tts))); } return o; } public int[] copycol (int p) { //int64 tts = GLib.get_monotonic_time(); int[] o = {}; for (int i = 0; i < dat.length; i++) { if (((i - p) % cols) == 0) { o += dat[i]; } } //int64 tte = GLib.get_monotonic_time(); //print("\tcopycol took %.2f microseconds\n",((double) (tte - tts))); return o; } public int getcell (int r, int c) { return ((int) ((r * cols) + c)); } public int[] getcol (int p) { //int64 tts = GLib.get_monotonic_time(); int[] o = {}; for (int i = 0; i < dat.length; i++) { if (((i - p) % cols) == 0) { o += i; } } //int64 tte = GLib.get_monotonic_time(); //print("\tgetcol took %.2f microseconds\n",((double) (tte - tts))); return o; } public void putcol (int p, int[] c) { //int64 tts = GLib.get_monotonic_time(); int n = 0; for (int i = 0; i < dat.length; i++) { if (((i - p) % cols) == 0) { dat[i] = c[n]; n += 1; } } //int64 tte = GLib.get_monotonic_time(); //print("\tputcol took %.2f microseconds\n",((double) (tte - tts))); } public void addcol (int p, int[] c) { int64 tts = GLib.get_monotonic_time(); int a = cols + 1; int j = (int) (dat.length / cols); int n = j - 1; dat.resize(dat.length + j); for (int i = (dat.length - 1); i >= 0; i--) { if (((i-p) % a) == 0) { dat[i] = c[n]; n -= 1; j -= 1; } else { dat[i] = dat[i-j]; } } cols += 1; int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t addcol took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t addcol took %.2f microseconds\n",((double) (tte - tts))); } } public void remcol (int p) { int64 tts = GLib.get_monotonic_time(); int j = (int) (dat.length / cols); int n = 0; int a = cols - 1; for (int i = p; i < (dat.length - j); i++) { if (((i - p) % a) == 0) { n += 1; } dat[i] = dat[i + n]; } dat.resize(dat.length - j); cols -= 1; int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t remcol took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t remcol took %.2f microseconds\n",((double) (tte - tts))); } } public void remcols (int[] p) { int64 tts = GLib.get_monotonic_time(); int j = (int) (dat.length / cols); j = j * p.length; int n = 0; int[] sp = p; snortint(sp); n = 0; for (int i = sp[0]; i < (dat.length - j); i++) { while (((i + n) % cols) in sp) { if ((i+n) >= dat.length) { break; } n += 1; } dat[i] = dat[i + n]; } dat.resize(dat.length - j); cols -= p.length; int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t remcols took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t remcols took %.2f microseconds\n",((double) (tte - tts))); } } // [ 0 1 2 3 ... 4 5 ] // [ 0 1 X X ] [ 2 3 X X ] [ 4 5 X X ] // +0+0 +2+2 +4+4 // diff = 2 // row = i / cols: 0/2=0, 1/2=0, 2/2=1, 3/2=1. 4/2=2, 5/2=2 // offs = row * dif: 0*2 = +0, 1*2 = +2, 2*2 = +4 // so: dat[5] copies to dat[9]: // 4-2 = 2 (new colsize - old colsize) // 5/2 = 2 (i/cols) // 2*2 = 4 (row * diff) // 4+5 = 9 (offset + i) public void resizecols(int t) { int64 tts = GLib.get_monotonic_time(); int rc = (dat.length / cols); int oc = dat.length; int d = t - cols; int p = 0; if (rc > 1) { if (d > 0) { dat.resize(t * rc); for (int i = (oc - 1); i >= 0; i-=1) { int r = (i / cols); int x = (i + (r*d)); if (x != i) { dat[i+(r*d)] = dat[i]; dat[i] = 0; } } } else { if (d < 0) { int j = 0; d = d.abs(); for ( int i = 0; i < oc; i++) { int r = (i / t); dat[i] = dat[i+(r*d)]; } dat.resize(t * rc); } } } cols = t; int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t resizecols took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t resizecols took %.2f microseconds\n",((double) (tte - tts))); } } public void swapcols (int s, int t) { int64 tts = GLib.get_monotonic_time(); int n = 0; int j = (int) (dat.length / cols); int[] ts = new int[j]; for (int i = 0; i < dat.length; i++) { if (((i - s) % cols) == 0) { ts[n] = i; n += 1; } } n = 0; for (int i = 0; i < dat.length; i++) { if (((i - t) % cols) == 0) { int td = dat[i]; dat[i] = dat[ts[n]]; dat[ts[n]] = td; n += 1; } } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t swapcols took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t swapcols took %.2f microseconds\n",((double) (tte - tts))); } } public void sortcolsbyrow (int r) { int64 tts = GLib.get_monotonic_time(); int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int[] l = copyrow(r); int n = l.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int[] k = copycol(i); int j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; putcol(j,copycol(j-g)); j -= g; } l[j] = t; putcol(j,k); } } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t sortcolsbyrow took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t sortcolsbyrow took %.2f microseconds\n",((double) (tte - tts))); } } public void sortcolsbyheader (string[] h) { int64 tts = GLib.get_monotonic_time(); int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = h.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { string t = h[i]; int[] k = copycol(i); int j = i; while (j >= g && (strcmp(h[j - g],t) > 0)) { h[j] = h[j - g]; putcol(j,copycol(j-g)); j -= g; } h[j] = t; putcol(j,k); } } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t sortcolsbyheader took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t sortcolsbyheader took %.2f microseconds\n",((double) (tte - tts))); } } public void filtercols (int k, int op, bool n) { int64 tts = GLib.get_monotonic_time(); // op 0 = starts_with // op 1 = ends_with // op 2 = contains // op 3 = equals // op 4 = greater than // op 5 = less than // n = not int[] ts = {}; for (int i = 0; i < dat.length; i++) { int ci = (i % cols); if (op == 0) { unichar aa = "%d".printf(dat[i]).get_char(0); unichar bb = "%d".printf(k).get_char(0); bool c = (aa == bb); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 1) { //string aa = "%d".printf(dat[i]); //aa = aa.substring(aa.length - 1); //string bb = "%d".printf(k); //bb = bb.substring(bb.length - 1); bool c = ((dat[i] % 10) == (k % 10)); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 2) { bool c = ("%d".printf(dat[i]).contains("%d".printf(k))); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 3) { bool c = (dat[i] == k); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 4) { bool c = (dat[i] > k); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 5) { bool c = (dat[i] < k); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } } int[] cids = new int[cols]; for (int i = 0; i < cols; i++) { cids[i] = i; } int[] rt = {}; for (int i = 0; i < cids.length; i++) { if ((cids[i] in ts) == false) { rt += cids[i]; } } if (rt.length > 0) { remcols(rt); } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t filtercols took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t filtercols took %.2f microseconds\n",((double) (tte - tts))); } } public string printdat (int ind, bool h) { //int64 tts = GLib.get_monotonic_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string dlm = " | "; int[] cw = new int[cols]; int pad = "%d".printf(cols).char_count(); for (int i = 0; i < cw.length; i++) { cw[i] = pad + 3; } string o = ""; string ln = ""; // get counts 1st for (int i = 0; i < dat.length; i++) { int r = (i % cols); string d = "%d".printf(dat[i]); int cc = 0; if (d != "") { cc = d.char_count() + 3; } cw[r] = int.max(cw[r],cc); } // print line for (int i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s+-".printf(tabs); } ln = "%s%s%*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-+\n".printf(ln); } } o = "%s%s".printf(o,ln); if (h) { // print header for (int i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } string hh = "%0*d".printf(pad,i); int cc = pad; o = "%s%s%*s%s".printf(o,dlm,(cw[i]-cc)," ",hh); if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); } // make rows for (int i = 0; i < dat.length; i++) { int r = (i % cols); int cc = "%d".printf(dat[i]).char_count(); dlm = " | "; if (r == 0) { dlm = "%s| ".printf(tabs); } o = "%s%s%*s%d".printf(o,dlm,(cw[r]-cc)," ",dat[i]); if (r == (cols - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); //int64 tte = GLib.get_monotonic_time(); //print("printdat took %.2f microseconds\n",((double) (tte - tts))); return o; } public void readstream (string p) { int64 tts = GLib.get_monotonic_time(); StringBuilder xx = new StringBuilder(""); string ff = Path.build_filename ("./", p); GLib.File fff = File.new_for_path(ff); if (fff.query_exists() == true) { try { uint8[] c; string e; fff.load_contents (null, out c, out e); xx.append((string) c); } catch (Error e) { print ("\tfailed to read %s: %s\n", fff.get_path(), e.message); } } else { print("file not found %s\n",ff); } int oo = xx.str.index_of(";"); string h = xx.str.substring(0,oo); cols = int.parse(h); oo = xx.str.index_of("\n"); xx.str = xx.str.substring(oo+1); string[] tdat = xx.str.split("\n"); xx.erase(0,-1); dat.resize(tdat.length); for (int s = 0; s < tdat.length; s++) { dat[s] = int.parse(tdat[s]); } tdat = new string[0]; int64 tte = GLib.get_monotonic_time(); print("readstream took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } public void setdat (int[] d, int c) { dat.resize(d.length); cols = c; for (int i = 0; i < d.length; i++) { dat[i] = d[i]; } } public MyDat (int l) { cols = l; dat = {}; } } void main() { MyDat m = new MyDat(5); m.setdat({0,1,2,3,4,0,1,2,3,4,0,1,2,3,4},5); print("\ndat:\n"); print("%s\n",m.printdat(1,false)); int[] cc = m.getcol(3); print("getcol(3) (get dat indices only):\n"); print("\t{"); for (int i = 0; i < (cc.length - 1); i++) { print("%d,",cc[i]); } print("%d}\n",cc[cc.length - 1]); cc = m.copycol(3); print("\ncopycol(3) (get data):\n"); print("\t{"); for (int i = 0; i < (cc.length - 1); i++) { print("%d,",cc[i]); } print("%d}\n",cc[cc.length - 1]); print("\nputcol(1,{7,7,7}) (overwrite):\n"); int[] nc = {7,7,7}; m.putcol(1,nc); print("%s\n",m.printdat(1,true)); print("\naddcol(3,{8,8,8}) (insert):\n"); nc = {8,8,8}; m.addcol(3,nc); print("%s\n",m.printdat(1,true)); print("\naddcol(6,{9,9,9}) (append):\n"); nc = {9,9,9}; m.addcol(6,nc); print("%s\n",m.printdat(1,true)); print("\nsortcolsbyrow(2):\n"); m.sortcolsbyrow(2); print("%s\n",m.printdat(1,true)); print("\nremcol(0):\n"); m.remcol(0); print("%s\n",m.printdat(1,true)); print("\nremcols({5,0,1}):\n"); nc = {5,0,1}; m.remcols(nc); print("%s\n",m.printdat(1,true)); print("\nswapcols(0,2):\n"); m.swapcols(0,2); print("%s\n",m.printdat(1,true)); // op 0 = starts_with // op 1 = ends_with // op 2 = contains // op 3 = equals // op 4 = greater than // op 5 = less than // n = not print("\nfiltercols(5,5,true): (not less than 5)\n"); m.filtercols(5,5,true); print("%s\n",m.printdat(1,true)); print("\nfiltercols(8,3,false): (is equal to 8)\n"); m.filtercols(8,3,false); print("%s\n",m.printdat(1,true)); print("\nmore cols...\n"); nc = {132,280,403}; m.addcol(1,nc); nc = {804,931,102}; m.addcol(2,nc); nc = {332,43,211}; m.addcol(2,nc); print("%s\n",m.printdat(1,true)); print("\nresizecols(6):\n"); m.resizecols(6); print("%s\n",m.printdat(1,true)); print("\nfiltercols(1,1,false): (ends with 1)\n"); m.filtercols(1,1,false); print("%s\n",m.printdat(1,true)); print("\nfiltercols(8,2,false): (contains 8)\n"); m.filtercols(8,2,false); print("%s\n",m.printdat(1,true)); print("\nstress test, 98737 x 95 ...\n"); print("loading csv into Dat..."); m.readstream("./dat.txt"); print("\nremcols({0,1,92,94,95}):\n"); nc = {0,1,92,94,95}; m.remcols(nc); print("\nfiltercols(0,1,true): (not ends with 0)\n"); m.filtercols(0,1,false); print("row 0:\n%s",m.printrow(1000)); print("\n\ncompare with dat[,]...\n"); int rows = (int) (m.dat.length / m.cols); int[,] dat = new int[rows,m.cols]; for (int r = 0; r < rows; r++) { for (int c = 0; c < m.cols; c++) { dat[r,c] = m.dat[m.getcell(r,c)]; } } int64 tts = GLib.get_monotonic_time(); int[,] tdat = new int[dat.length[0], dat.length[1]]; int n = 0; for (int c = 0; c < dat.length[1]; c++) { if (c in nc) { n += 1; } for (int r = 0; r < dat.length[0]; r++) { tdat[r,(c-n)] = dat[r,c]; } } dat = tdat; tdat = new int [0,0]; int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t removecols from 2d-array took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t removecols from 2d-array took %.2f microseconds\n",((double) (tte - tts))); } }
#+RESULTS:
Compilation succeeded - 3 warning(s) dat: +------+------+------+------+------+ | 0 | 1 | 2 | 3 | 4 | | 0 | 1 | 2 | 3 | 4 | | 0 | 1 | 2 | 3 | 4 | +------+------+------+------+------+ getcol(3) (get dat indices only): {3,8,13} copycol(3) (get data): {3,3,3} putcol(1,{7,7,7}) (overwrite): +------+------+------+------+------+ | 0 | 1 | 2 | 3 | 4 | +------+------+------+------+------+ | 0 | 7 | 2 | 3 | 4 | | 0 | 7 | 2 | 3 | 4 | | 0 | 7 | 2 | 3 | 4 | +------+------+------+------+------+ addcol(3,{8,8,8}) (insert): addcol took 0.00 microseconds +------+------+------+------+------+------+ | 0 | 1 | 2 | 3 | 4 | 5 | +------+------+------+------+------+------+ | 0 | 7 | 2 | 8 | 3 | 4 | | 0 | 7 | 2 | 8 | 3 | 4 | | 0 | 7 | 2 | 8 | 3 | 4 | +------+------+------+------+------+------+ addcol(6,{9,9,9}) (append): addcol took 0.00 microseconds +------+------+------+------+------+------+------+ | 0 | 1 | 2 | 3 | 4 | 5 | 6 | +------+------+------+------+------+------+------+ | 0 | 7 | 2 | 8 | 3 | 4 | 9 | | 0 | 7 | 2 | 8 | 3 | 4 | 9 | | 0 | 7 | 2 | 8 | 3 | 4 | 9 | +------+------+------+------+------+------+------+ sortcolsbyrow(2): copyrow took 0.00 microseconds sortcolsbyrow took 25.00 microseconds +------+------+------+------+------+------+------+ | 0 | 1 | 2 | 3 | 4 | 5 | 6 | +------+------+------+------+------+------+------+ | 0 | 2 | 3 | 4 | 7 | 8 | 9 | | 0 | 2 | 3 | 4 | 7 | 8 | 9 | | 0 | 2 | 3 | 4 | 7 | 8 | 9 | +------+------+------+------+------+------+------+ remcol(0): remcol took 0.00 microseconds +------+------+------+------+------+------+ | 0 | 1 | 2 | 3 | 4 | 5 | +------+------+------+------+------+------+ | 2 | 3 | 4 | 7 | 8 | 9 | | 2 | 3 | 4 | 7 | 8 | 9 | | 2 | 3 | 4 | 7 | 8 | 9 | +------+------+------+------+------+------+ remcols({5,0,1}): remcols took 3.00 microseconds +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 4 | 7 | 8 | | 4 | 7 | 8 | | 4 | 7 | 8 | +------+------+------+ swapcols(0,2): swapcols took 0.00 microseconds +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 8 | 7 | 4 | | 8 | 7 | 4 | | 8 | 7 | 4 | +------+------+------+ filtercols(5,5,true): (not less than 5) remcols took 0.00 microseconds filtercols took 6.00 microseconds +------+------+ | 0 | 1 | +------+------+ | 8 | 7 | | 8 | 7 | | 8 | 7 | +------+------+ filtercols(8,3,false): (is equal to 8) remcols took 0.00 microseconds filtercols took 7.00 microseconds +------+ | 0 | +------+ | 8 | | 8 | | 8 | +------+ more cols... addcol took 0.00 microseconds addcol took 0.00 microseconds addcol took 0.00 microseconds +------+--------+--------+--------+ | 0 | 1 | 2 | 3 | +------+--------+--------+--------+ | 8 | 132 | 332 | 804 | | 8 | 280 | 43 | 931 | | 8 | 403 | 211 | 102 | +------+--------+--------+--------+ resizecols(6): resizecols took 1.00 microseconds +------+--------+--------+--------+------+------+ | 0 | 1 | 2 | 3 | 4 | 5 | +------+--------+--------+--------+------+------+ | 8 | 132 | 332 | 804 | 0 | 0 | | 8 | 280 | 43 | 931 | 0 | 0 | | 8 | 403 | 211 | 102 | 0 | 0 | +------+--------+--------+--------+------+------+ filtercols(1,1,false): (ends with 1) remcols took 1.00 microseconds filtercols took 5.00 microseconds +--------+--------+ | 0 | 1 | +--------+--------+ | 332 | 804 | | 43 | 931 | | 211 | 102 | +--------+--------+ filtercols(8,2,false): (contains 8) remcols took 1.00 microseconds filtercols took 7.00 microseconds +--------+ | 0 | +--------+ | 804 | | 931 | | 102 | +--------+ stress test, 98737 x 95 ... loading csv into Dat...file not found ././dat.txt readstream took 0.003700 seconds remcols({0,1,92,94,95}):
COLRESIZE:vala:
// resize a column in a 1d array to t rows, adding blanks as required // by c.p.brown 2024 public class MyDat : Object { public int dsz; // column count public string[] dat; // data public void coladd (int p, string[] c) { int64 tts = GLib.get_monotonic_time(); int a = dsz + 1; int j = (int) (dat.length / dsz); int n = j - 1; dat.resize(dat.length + j); for (int i = (dat.length - 1); i >= 0; i--) { if (((i-p) % a) == 0) { dat[i] = c[n]; n -= 1; j -= 1; } else { dat[i] = dat[i-j]; } } dsz += 1; int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t addcol took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t addcol took %.2f microseconds\n",((double) (tte - tts))); } } public void rowspam (int r, int c) { dat.resize(r * c); dsz = c; GLib.Rand rando = new GLib.Rand(); rando.set_seed(5); for (int i = 0; i < dat.length; i++) { //int rch = rando.int_range(65,90); int rch = i / dsz; dat[i] = "%d".printf(rch); //dat[i] = "%s".printf(((unichar) rch).to_string()); } print("rowspam complete.\n"); } public void rowappend (string[] c) { int64 tts = GLib.get_monotonic_time(); int p = dat.length; dat.resize(dat.length + dsz); int l = dat.length; int m = c.length; if (c == null) { for (int i = p; i < l; i++) { dat[i] = ""; } } else { for (int i = p; i < l; i++) { if ((i-p) < dsz) { if ((i-p) < m) { dat[i] = c[i - p]; } else { dat[i] = ""; } } } } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t rowappend took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t rowappend took %.2f microseconds\n",((double) (tte - tts))); } } public void colappendat (int p, string[] c) { int a = dsz + 1; int j = (int) (dat.length / dsz); int n = j - 1; dat.resize(dat.length + j); for (int i = (dat.length - 1); i >= 0; i--) { int r = (int) (i / dsz); if (((i-p) % a) == 0) { dat[i] = c[n]; n -= 1; j -= 1; } else { dat[i] = dat[i-j]; } } dsz += 1; } // +---+---------------+ // | T | 4 (new cols) | // | S | 2 (cols) | // | D | T - S (2) | // | R | A.length / S | // | J | T * R | // +---+---------------+ // // +---+----------+-------+-------+-------+-------+-------+-------+ // | A.RESIZE(J) | 00 01 | 02 03 | 04 05 | XX XX | XX XX | XX XX | // +---+----------+-------+-------+-------+-------+-------+-------+ // | I | (index) | 0 1 | 2 3 | 4 5 | 5 7 | 8 9 | 10 11 | // | R | I / S | 0 0 | 1 1 | 2 2 | | | | // | O | R * D | 0 0 | 2 2 | 4 4 | | | | // | X | I + O | 0 1 | 4 5 | 8 9 | | | | // +---+----------+-------+-------+-------+-------+-------+-------+ // +--------------+-------+-------+-v--v--+-------+-v--v--+-------+ // | A[X] = A[I] | 00 01 | XX XX | 02 03 | XX XX | 04 05 | XX XX | // +--------------+-------+-------+-------+-------+-------+-------+ // | cols = 4 | 0 1 2 3 | 4 5 6 7 | 8 9 10 11 | // +--------------+---------------+---------------+---------------+ // +---+------------------+ // | T | 4 (new cols) | // | S | 3 (cols) | // | D | T - S (1) | // | W | A.length / S (2) | // | J | T * W (8) | // +---+------------------+ // // +--------------+----------+----------+-------+ // | A.RESIZE(J) | 00 01 02 | 03 04 05 | XX XX | // +---+----------+----------+----------+-------+ // +---+----------+----------+----------+ // | I | (index) | 0 1 2 | 3 4 5 | // | R | I / S(3) | 0 0 0 | 1 1 1 | // | O | R * D(1) | 0 0 0 | 1 1 1 | // | X | I + O | 0 1 2 | 4 5 6 | // +---+----------+----------+----------+ // +--------------+-------------+-------------+ // | A[X] = A[I] | 00 01 02 XX | 03 04 05 XX | // +--------------+-------------+-------------+ // | index | 0 1 2 3 | 4 5 6 7 | // +--------------+-------------+-------------+ public void colresize (int t) { int64 tts = GLib.get_monotonic_time(); int rc = (dat.length / dsz); int oc = dat.length; int d = t - dsz; int p = 0; if (rc > 1) { if (d > 0) { dat.resize(t * rc); for (int i = oc; i < dat.length; i++) { dat[i] = ""; } for (int i = (oc - 1); i >= 0; i-=1) { int r = (i / dsz); int x = (i + (r*d)); if (x != i) { dat[x] = dat[i]; dat[i] = ""; } } } else { if (d < 0) { int j = 0; for ( int i = 0; i < oc; i++) { int r = (i / dsz); int x = (i+(r*d)); if (i != x) { dat[x] = dat[i]; } } dat.resize(t * rc); } } } dsz = t; int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t colresize took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t colresize took %.2f microseconds\n",((double) (tte - tts))); } } public string rowprint(int w) { //print("rowprint started...\n"); int e = (dat.length / dsz); int r = w; if ((r >= 0) && (r < e)) { e = r; } int a = (e * dsz) - dsz; int b = (a + dsz); //print("row is %d\n",e); // print("dat.length is %d\n",dat.length); //print("dat.dsz is %d\n",dsz); // print("dat.length / dsz is %.2f\n",(((double) dat.length) / ((double) dsz))); //print("range is dat[%d] to dat[%d]\n",a,(b-1)); string o = "row %d is {".printf(r); for (int i = a; i < (b-1); i++) { o = (o + dat[i] + ","); } o = (o + dat[b-1] + "}\n"); //print("rowprint complete.\n"); return o; } public string datreport (int ind, bool h) { //int64 tts = GLib.get_monotonic_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string dlm = " | "; int[] cw = new int[dsz]; int pad = "%d".printf(dsz).char_count(); for (int i = 0; i < cw.length; i++) { cw[i] = pad; } string o = ""; string ln = ""; // get counts 1st for (int i = 0; i < dat.length; i++) { int r = (i % dsz); string d = "%s".printf(dat[i]); int cc = 0; if (d != "") { cc = d.char_count(); } cw[r] = int.max(cw[r],cc); } // print line for (int i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s+-".printf(tabs); } ln = "%s%s%*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-+\n".printf(ln); } } o = "%s%s".printf(o,ln); if (h) { // print header for (int i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } string hh = "%0*d".printf(pad,i); int cc = pad; o = "%s%s%.*s%s".printf(o,dlm,int.max(0,(cw[i]-cc))," ",hh); if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); } // make rows for (int i = 0; i < dat.length; i++) { int r = (i % dsz); int cc = dat[i].char_count(); dlm = " | "; if (r == 0) { dlm = "%s| ".printf(tabs); } o = "%s%s%.*s%s".printf(o,dlm,int.max(0,(cw[r]-cc))," ",dat[i]); if (r == (dsz - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); //int64 tte = GLib.get_monotonic_time(); //print("printdat took %.2f microseconds\n",((double) (tte - tts))); return o; } public MyDat () { dat = {}; } } void main() { MyDat m = new MyDat(); m.dsz = 5; m.rowappend({"0","1","2","3","4"}); m.rowappend({"1","A","H","U","W"}); m.rowappend({"2","B","J","I","X"}); m.rowappend({"3","C","K","O","Y"}); m.rowappend({"4","D","L","P","Z"}); print(m.datreport(1,false)); m.colresize(3); print(m.datreport(1,false)); m.colresize(6); print(m.datreport(1,false)); m.colresize(1); print(m.datreport(1,false)); m.colresize(5); print(m.datreport(1,false)); print("setting dat rows to %d, cols to %d...\n",674,6); m.rowspam(674,6); print(m.rowprint(-1)); m.colresize(7); print(m.rowprint(-1)); print(m.rowprint(10)); }
#+RESULTS:
Compilation succeeded - 3 warning(s) rowappend took 2.00 microseconds rowappend took 0.00 microseconds rowappend took 0.00 microseconds rowappend took 1.00 microseconds rowappend took 0.00 microseconds +---+---+---+---+---+ | 0 | 1 | 2 | 3 | 4 | | 1 | A | H | U | W | | 2 | B | J | I | X | | 3 | C | K | O | Y | | 4 | D | L | P | Z | +---+---+---+---+---+ colresize took 1.00 microseconds +---+---+---+ | 0 | 1 | 2 | | 1 | A | H | | 2 | B | J | | 3 | C | K | | 4 | D | L | +---+---+---+ colresize took 2.00 microseconds +---+---+---+---+---+---+ | 0 | 1 | 2 | | | | | 1 | A | H | | | | | 2 | B | J | | | | | 3 | C | K | | | | | 4 | D | L | | | | +---+---+---+---+---+---+ colresize took 1.00 microseconds +---+ | 0 | | 1 | | 2 | | 3 | | 4 | +---+ colresize took 2.00 microseconds +---+---+---+---+---+ | 0 | | | | | | 1 | | | | | | 2 | | | | | | 3 | | | | | | 4 | | | | | +---+---+---+---+---+ setting dat rows to 674, cols to 6... rowspam complete. row -1 is {673,673,673,673,673,673} colresize took 219.00 microseconds row -1 is {673,673,673,673,673,673,} row 10 is {9,9,9,9,9,9,}
COLUMNID:vala:
// extract column number from spreadsheet codes, // assuming A->Z, then AA->ZZ, then AAA->ZZZ, and so-on // this is derivative work from various sources void main () { string[] s = { "A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z","AA", "BB", "AZ", "ZZ", "AAA", "CR", "CQ" }; for (int i = 0; 0 < s.length; i++) { int colnum = 0; for (int c = 0; c < s[i].char_count(); c++) { int cc = (((int) s[i].get_char(c)) - 65); // A = 65 colnum = colnum * 26; // nth letter is nth * number of letters colnum += cc + 1; } print("%s = %d\n",s[i],colnum); } }
#+RESULTS:
A = 1 B = 2 C = 3 D = 4 E = 5 F = 6 G = 7 H = 8 I = 9 J = 10 K = 11 L = 12 M = 13 N = 14 O = 15 P = 16 Q = 17 R = 18 S = 19 T = 20 U = 21 V = 22 W = 23 X = 24 Y = 25 Z = 26 AA = 27 BB = 54 AZ = 52 ZZ = 702 AAA = 703 CR = 96 CQ = 95 (null) = 0 (null) = 0
COMPACT:vala:

Reduce multiple contiguous whitespaces to one single space.
Considers newline a whitespace.

// compact string // by cpbrown 2024 string compactstring (string s, bool n) { int64 tts = GLib.get_monotonic_time(); // n = remove newlines string compacted = ""; if (n) { for (int i = 0; i < s.char_count(); i++) { unichar tc = s.get_char(s.index_of_nth_char(i)); if (!tc.isspace()) { compacted = (compacted + tc.to_string()); continue; } else { unichar pc = 'A'; if (i > 0) { pc = s.get_char(s.index_of_nth_char(i-1)); } if (pc.isspace()) { continue; } tc = ' '; compacted = (compacted + tc.to_string()); } } } else { for (int i = 0; i < s.char_count(); i++) { unichar tc = s.get_char(s.index_of_nth_char(i)); if (tc == '\n') { compacted = (compacted + tc.to_string()); continue; } if (!tc.isspace()) { compacted = (compacted + tc.to_string()); continue; } else { unichar pc = 'A'; if (i > 0) { pc = s.get_char(s.index_of_nth_char(i-1)); } if (pc != '\n') { if (pc.isspace()) { continue; } } tc = ' '; compacted = (compacted + tc.to_string()); } } } compacted = compacted.strip(); int64 tte = GLib.get_monotonic_time(); print("compact %.2f microseconds\n",((double) (tte - tts))); return compacted; } void main() { GLib.Intl.setlocale(ALL,""); string s = " a messy aut零parsed\tline\t\t\n of\n «text» "; print("original string is:\n\"%s\"\n\n",s); string compacted = compactstring(s,true); print("\ncompacted string is:\n\"%s\"\n\n",compacted); compacted = compactstring(s,false); print("\ncompacted string is:\n\"%s\"\n",compacted); }
#+RESULTS:
original string is: " a messy aut零parsed line of «text» " compact 25.00 microseconds compacted string is: "a messy aut零parsed line of «text»" compact 17.00 microseconds compacted string is: "a messy aut零parsed line of «text»"
CONCAT:vala:
// speed difference between a+b, a.concat(b) and "%s%s".printf(a,b) // also: trying out GLib.Timer // by c.p.brown 2024 void snortdouble (double[] l) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = l.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { double t = l[i]; int j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; j -= g; } l[j] = t; } } } double getmediandouble (double[] n) { int l = n.length; double m = -1.0; if ((l & 1) == 0) { int aa = ((int) (l * 0.5)) - 1; int bb = aa + 1; m = (n[aa] + n[bb]) * 0.5; } else { int aa = ((int) (l * 0.5)); m = n[aa]; } return m; } void main() { string aa = ""; string bb = ""; string testfile = "cpbrown_notes.org"; string ff = Path.build_filename ("./", testfile); GLib.File og = File.new_for_path(ff); if (og.query_exists() == true) { try { uint8[] c; string e; og.load_contents (null, out c, out e); aa = (string) c; bb = (string) c; } catch (Error e) { print ("\tfailed to read %s: %s\n", og.get_path(), e.message); } } print("\naa & bb are copies of this orgfile\n\n"); double[] tca = new double[10]; double[] tcb = new double[10]; double[] tcc = new double[10]; double vta = 0.0; double vtb = 0.0; double vtc = 0.0; Timer tt = new Timer (); for (int i = 0; i < 10; i ++) { tt.reset(); double uu; double dd; tt.start(); string cc = aa + bb; tt.stop(); dd = tt.elapsed(out uu); vta += dd; tca[i] = dd; tt.reset(); tt.start(); string ee = aa.concat(bb); tt.stop(); dd = tt.elapsed(out uu); vtb += dd; tcb[i] = dd; tt.reset(); tt.start(); string gg = "%s%s".printf(aa,bb); tt.stop(); dd = tt.elapsed(out uu); vtc += dd; tcc[i] = dd; } snortdouble(tca); snortdouble(tcb); snortdouble(tcc); double mtca = getmediandouble(tca); double mtcb = getmediandouble(tcb); double mtcc = getmediandouble(tcc); print("\n1st run\n"); print("aa + bb %.2f microseconds\n",(tca[0] * 1000000.0)); print("aa.concat(bb) %.2f microseconds\n",(tcb[0] * 1000000.0)); print("\"%%s%%s\".prinft(aa,bb) %.2f microseconds\n",(tcc[0] * 1000000.0)); print("\naverage of 10\n"); print("aa + bb %.2f microseconds\n",((vta * 0.1) * 1000000.0)); print("aa.concat(bb) %.2f microseconds\n",((vtb * 0.1) * 1000000.0)); print("\"%%s%%s\".prinft(aa,bb) %.2f microseconds\n",((vtc * 0.1) * 1000000.0)); print("\nmedian of 10\n"); print("aa + bb %.2f microseconds\n",(mtca * 1000000.0)); print("aa.concat(bb) %.2f microseconds\n",(mtcb * 1000000.0)); print("\"%%s%%s\".prinft(aa,bb) %.2f microseconds\n",(mtcc * 1000000.0)); print("\nugh great, I've been using printfs everywhere...\n"); }
#+RESULTS:
Compilation succeeded - 3 warning(s) aa & bb are copies of this orgfile 1st run aa + bb 41.00 microseconds aa.concat(bb) 150.00 microseconds "%s%s".prinft(aa,bb) 206.00 microseconds average of 10 aa + bb 66.30 microseconds aa.concat(bb) 163.30 microseconds "%s%s".prinft(aa,bb) 229.50 microseconds median of 10 aa + bb 41.50 microseconds aa.concat(bb) 153.00 microseconds "%s%s".prinft(aa,bb) 207.50 microseconds ugh great, I've been using printfs everywhere...
CONDITIONAL:vala:
int main (string[] args) { string[] t = {"0", "01", "0123", "01234"}; foreach (string s in t) { if (s.length > 2 && s.substring(0,3) == "012") { print("checked length: %s\n",s); } } return 0; }
#+RESULTS:
checked length: 0123 checked length: 01234 state = 1 state = 2 state = 0 state = 1 state = 2
CONTIGUOUS:vala:
// break int array into contiguous sequences // by c.p.brown 2023 struct packet { int[] items; } packet[] contigiousints (int[] n) { packet[] o = {}; packet p = packet(); p.items += n[0]; o += p; for (int i = 1; i < n.length; i++) { if ((n[i] - n[i-1]) > 1) { packet b = packet(); b.items += n[i]; o += b; } else { o[o.length - 1].items += n[i]; } } return o; } void main() { int[] n = {2, 4, 5, 6, 9, 11, 12, 18, 19, 20}; packet[] cblocks = contigiousints(n); for (int c = 0; c < cblocks.length; c++) { print("cblock %d has %d members\n",c,cblocks[c].items.length); print("\t{"); for (int i = 0; i < cblocks[c].items.length; i++) { if (i < (cblocks[c].items.length - 1)) { print("%d,",cblocks[c].items[i]); } else { print("%d}\n",cblocks[c].items[i]); } } } }
#+RESULTS:
cblock 0 has 1 members {2} cblock 1 has 3 members {4,5,6} cblock 2 has 1 members {9} cblock 3 has 2 members {11,12} cblock 4 has 3 members {18,19,20}
CURRENCY:vala:
void main() { string s = "AUD$312.235"; print("double.parse(\"%s\") is %f\n",s,double.parse(s)); unichar[] sym = {'$','₹','¥','฿','₫','£','€'}; int ii = 0; for (int i = 0; i < s.length; i++) { if (s[i] in sym) { ii = (i + 1); break; } } string nosym = s.substring(ii); print("\"%s\".substring(%d) is %s\n",s,ii,nosym); // allgood, but too easy, try a fuckier format: s = "XYZ -500'000,23€"; print("\ntest string %s has %d chars\n",s,s.length); string p = ""; for (int i = 0; i < s.length; i++) { unichar c = s.get_char(i); string cs = c.to_string(); if (c.isdigit()) { p += cs; } if (c == '-') { p += cs; } if (c == '+') { p += cs; } if (c == '.') { p += cs; } if (c == ',') { p += cs; } if (c == '\'') { p += cs; } } print("\nmanually parsed string is %s\n",p); print("double.parse(\"%s\") is %f\n",s,double.parse(p)); // the punctuation breaks it, try one last thing... p = ""; for (int i = 0; i < s.length; i++) { unichar c = s.get_char(i); string cs = c.to_string(); if (c.isdigit()) { p += cs; } if (c == '-') { p += cs; } if (c == '+') { p += cs; } if (c == '.') { p += cs; } if (c == ',') { p += "."; } //if (c == '\'') { p += ""; } } print("\nmanually parsed string with edits is %s\n",p); print("double.parse(\"%s\") is %f\n",s,double.parse(p)); }
#+RESULTS:
double.parse("AUD$312.235") is 0.000000 "AUD$312.235".substring(4) is 312.235 test string XYZ -500'000,23? has 18 chars manually parsed string is -500'000,23 double.parse("XYZ -500'000,23?") is -500.000000 manually parsed string with edits is -500000.23 double.parse("XYZ -500'000,23?") is -500000.230000
DATE2:wip::vala:

second attempt at a generic setdate()
Better provision for correction in a date sequence.
Still contains errors, this will be replaced at some point.

// generic date parser, 2nd attempt // by c.p.brown 2023 // // assumptions: // prefers iso8601 date format, or reverse iso8601 if year is last // ass-backwards U.S. dates will require a user override, // unless the day is greater than 12 and in 2nd position, // in which case the override applies automatically // dates come in 3 parts // year is never 2nd // month is never last // if year is 1st, month is 2nd, day is last // if day is last, month is 2nd, year is 1st // 4-digit numbers are years // strings may be months // weekdays, prepositions, whiggles & time-notation can be safely filtered-out // years in the outer loop tend to be the same or close together += 5 or less, eg; 20,20,21,25,25,26... // deifinte dates like "2023 feb 20" can be used to correct ambigious dates in an outer loop, by establishing the format, "Y b d" in this case // so the format should be returned along with the date and an ambiguity check: // mydate { // value = {2020,2,20}; // format = {0,1,2}; // certainty = 3; // } struct mydate { int[] value; int[] format; int certainty; } mydate setanydate(string s, int[] knownformat, bool spew) { mydate ret = mydate(); ret.value = {-1,-1,-1}; ret.format = {-1,-1,-1}; ret.certainty = -1; string[] longmonths = {"january","february","march","april","may","june","july","august","september","october","november","december"}; string[] shortmonths = {"jan","feb","mar","apr","may","jun","jul","aug","sep","oct","nov","dec"}; // output ymd int[] thedate = {-1,-1,-1}; // input positions for y/m/d, // eg: {2,1,0} // y m d ... {20,10,23} // 0 1 2 ... year = date[format[0]] = 23 // month = date[format[1]] = 10 // day = date[format[2]] = 20 int[] theformat = {-1,-1,-1}; int defi = 0; bool preformat = false; if ((-1 in knownformat) == false) { theformat = knownformat; preformat = true; } int[] datepos = {-1,-1,-1}; string cs = s; cs = cs.down(); cs = cs.replace(" of",""); cs = cs.replace(" am",""); cs = cs.replace(" pm",""); cs = cs.replace("am",""); cs = cs.replace("pm",""); string[] timepunc = {"gmt","utc",":","+"}; foreach (string tpu in timepunc) { if (cs.contains(tpu)) { string[] csparts = cs.split(" "); if (csparts.length > 1) { string[] ncs = {}; for (int i = 0; i < csparts.length; i++) { if (csparts[i].contains(tpu)) { if (csparts[i].contains("t")) { string[] bcsparts = csparts[i].split("t"); if (bcsparts.length == 2) { ncs += bcsparts[0]; if (bcsparts[1].contains(tpu) == false) { ncs += bcsparts[1]; } } } } else { if (csparts[i].contains(tpu) == false) { ncs += csparts[i]; } } } if (ncs.length > 0) { cs = string.joinv(" ",ncs); } } else { // its probably an iso 8601 with time, so check for single "t", then split, keep [0] string[] isop = cs.split("t"); if (isop.length == 2) { if (isop[0].contains(tpu) == false) { cs = isop[0]; } else { cs = isop[1]; } } } } } string[] wkday = { "mon", "tue", "wed", "thu", "fri", "sat", "sun" }; foreach (string wkd in wkday) { if (cs.contains(wkd)) { string[] csparts = cs.split(" "); if (csparts.length > 1) { string[] ncs = {}; for (int i = 0; i < csparts.length; i++) { if (csparts[i].contains(wkd) == false) { ncs += csparts[i]; } } if (ncs.length > 0) { cs = string.joinv(" ",ncs); } } } } cs = cs.replace(",",""); cs = cs.replace("st",""); cs = cs.replace("nd",""); cs = cs.replace("rd",""); cs = cs.replace("th",""); string[] dateparts = {}; string[] delimiters = {" ","-","/"}; foreach(string l in delimiters) { dateparts = cs.split(l); if (dateparts.length == 3) { break; } } // handle sortable numbers: yyyymmdd and yymmdd if (dateparts.length == 1) { string[] dpts = new string[3]; if (cs.char_count() == 8) { dpts[0] = cs.substring(0,4); dpts[1] = cs.substring(4,2); dpts[2] = cs.substring(6,2); preformat = true; theformat = {0,1,2}; dateparts = dpts; } if (cs.char_count() == 6) { dpts[0] = cs.substring(0,2); dpts[1] = cs.substring(2,2); dpts[2] = cs.substring(4,2); preformat = true; theformat = {0,1,2}; dateparts = dpts; } } if (dateparts.length == 3) { if (preformat) { thedate[0] = int.parse(dateparts[theformat[0]]); if (dateparts[theformat[1]].char_count() >= 3) { int threec = int.parse(dateparts[theformat[1]]); if (threec == 0) { for (int m = 0; m < longmonths.length; m++) { if (strcmp(longmonths[m], dateparts[theformat[1]]) == 0) { thedate[1] = (m+1); } } if (thedate[1] == -1) { for (int m = 0; m < shortmonths.length; m++) { if (strcmp(shortmonths[m],dateparts[theformat[1]]) == 0) { thedate[1] = (m+1); } } } } } else { thedate[1] = int.parse(dateparts[theformat[1]]); } thedate[2] = int.parse(dateparts[theformat[2]]); } else { // 1st round: get obvious date parts that don't depend on each other for (int p = 0; p < dateparts.length; p++) { if (p != 1) { if (dateparts[p].char_count() == 4) { int fourc = int.parse(dateparts[p]); if((fourc > 1700) && (fourc < 2070)) { print("\tdateparts[%d] %d is year (4-num & > 1700 & < 2070)\n",p,fourc); if(thedate[0] == -1) { thedate[0] = fourc; datepos[0] = p; defi += 1; } } else { if (spew) { print("4-number month/day %d (%s) not supported, skipping\n",fourc,dateparts[p]); } } } } int twoc = int.parse(dateparts[p]); if (twoc > 0) { if (twoc > 31) { if (thedate[0] == -1) { print("\tdateparts[%d] %d is year (n > 31)\n",p,twoc); thedate[0] = twoc; datepos[0] = p; defi += 1; } } } if (p != 2) { if (dateparts[p].char_count() >= 3) { int threec = int.parse(dateparts[p]); if (threec == 0) { for (int m = 0; m < longmonths.length; m++) { if (strcmp(longmonths[m], dateparts[p]) == 0) { if (thedate[1] == -1) { print("\tdateparts[%d] %d is month (found string %s)\n",p,(m+1),dateparts[p]); thedate[1] = (m+1); datepos[1] = p; defi += 1; break; } } } if (thedate[1] == -1) { for (int m = 0; m < shortmonths.length; m++) { if (strcmp(shortmonths[m], dateparts[p]) == 0) { if (thedate[1] == -1) { print("\tdateparts[%d] %d is month (found string %s)\n",p,(m+1),dateparts[p]); thedate[1] = (m+1); datepos[1] = p; defi += 1; break; } } } } if (thedate[1] != -1) { print("\t\tthedate[1] (month) is %d\n",thedate[1]); } } else { if (spew) { print("\t\t3-number month/day %s not supported, skipping\n",dateparts[p]); } } } } } // 2nd round: get dateparts that depend on the position of other dateparts // this may require several iterations int itr = 0; while (-1 in thedate) { if (itr > 9) { break; } for (int p = 0; p < dateparts.length; p++) { if (dateparts[p].char_count() <= 2) { int twoc = int.parse(dateparts[p]); // day is last, so year should be 1st if ((datepos[2] == 2) && (p == 0)) { if (thedate[0] == -1) { print("\tdateparts[%d] %d is year (day is last, i am first)\n",p,twoc); thedate[0] = twoc; datepos[0] = p; defi += 1; } } // year is last, month is 2nd if ((p == 1) && (datepos[0] == 2)) { if ((thedate[1] == -1) && (twoc <= 12)) { print("\tdateparts[%d] %d is month (year is last, i am 2nd)\n",p,twoc); thedate[1] = twoc; datepos[1] = p; defi += 1; } } // year is 1st, day is last if (datepos[0] == 0) { if( thedate[2] == -1) { print("\tdateparts[%d] %d is day (year is @%d, i am last)\n",p,twoc,datepos[0]); thedate[2] = twoc; datepos[2] = p; defi += 1; } } // year is taken, n > 12 if (twoc > 12) { if (thedate[0] != -1) { if( thedate[2] == -1) { print("\tdateparts[%d] %d is day (n > 12 & year is taken)\n",p,twoc); thedate[2] = twoc; datepos[2] = p; defi += 1; } } } // year is taken, day is taken if ((thedate[0] != -1) && (thedate[2] != -1)) { if ((thedate[1] == -1) && ((p in datepos) == false) && (twoc <= 12)) { print("\tdateparts[%d] %d is month (year + day are taken)\n",p,twoc); thedate[1] = twoc; datepos[1] = p; defi += 1; } } // year is taken, month is taken if ((thedate[0] != -1) && (thedate[1] != -1)) { if ((thedate[2] == -1) && ((p in datepos) == false)) { print("\tdateparts[%d] %d is day (year & month are taken)\n",p,twoc); thedate[2] = twoc; datepos[2] = p; defi += 1; } } } } itr += 1; } // final round: take a wild fucking guess int[] tw = new int[3]; tw[0] = int.parse(dateparts[0]); tw[1] = int.parse(dateparts[1]); tw[2] = int.parse(dateparts[2]); int dd = 0; int ss = 0; dd = int.max(int.max(tw[0],tw[1]),tw[2]); ss = int.min(int.min(tw[0],tw[1]),tw[2]); itr = 0; while (-1 in thedate) { if (itr > 9) { break; } for (int p = 0; p < dateparts.length; p++) { if (dateparts[p].char_count() <= 2) { int twoc = int.parse(dateparts[p]); if (twoc > 0) { if((thedate[0] == -1) && (datepos[0] == -1) && (twoc == dd)) { print("\tdateparts[%d] %d is year? (fallback)\n",p,twoc); thedate[0] = twoc; datepos[0] = p; continue; } if((thedate[2] == -1) && (datepos[2] == -1)) { if ((p == 0) && (twoc <= 31)) { print("\tdateparts[%d] %d is day? (fallback)\n",p,twoc); thedate[2] = twoc; datepos[2] = p; continue; } } if((thedate[1] == -1) && (datepos[1] == -1) && (twoc == ss)) { if ((p == 1) && (twoc <= 12)) { print("\tdateparts[%d] %d is month? (fallback)\n",p,twoc); thedate[1] = twoc; datepos[1] = p; continue; } } // year is taken, day is taken if ((thedate[0] != -1) && (thedate[2] != -1)) { if ((thedate[1] == -1) && ((p in datepos) == false) && (twoc <= 12)) { print("\tdateparts[%d] %d is month (year + day are taken)\n",p,twoc); thedate[1] = twoc; datepos[1] = p; defi += 1; } } // year is taken, month is taken if ((thedate[0] != -1) && (thedate[1] != -1)) { if ((thedate[2] == -1) && ((p in datepos) == false)) { print("\tdateparts[%d] %d is day (year & month are taken)\n",p,twoc); thedate[2] = twoc; datepos[2] = p; defi += 1; } } // month is taken, day is taken if ((thedate[1] != -1) && (thedate[2] != -1)) { if ((thedate[0] == -1) && ((p in datepos) == false)) { print("\tdateparts[%d] %d is year (month & day are taken)\n",p,twoc); thedate[0] = twoc; datepos[0] = p; defi += 1; } } } } } itr += 1; } } print("\noriginal date string : %s\n",s); print("year is %d\n",thedate[0]); print("month is %d\n",thedate[1]); print("day is %d\n",thedate[2]); if (!preformat) { print("certainty is %d\n",defi); } print("\n"); ret.value = thedate; ret.format = datepos; ret.certainty = defi; } return ret; } void main() { string[] q = { "9/10/2023", "10/9/2023", "20/11/2023", "11/20/2023", "23-9-10", "9 10 23", "230910", "20231102", "23111", "11 30 23", "10 sep 23", "20 Feb 2020", "sep 10 2023", "september 10th, 2023", "10th of September, 2023", "2023-10-09T22:07:23+00:00", "2023-10-09 22:07:23 +00:00", "Tuesday October 10th, 2023", "9:50AM Tuesday October 10th, 2023" }; int[] kf = {-1,-1,-1}; print("\nbatch test on a random date formats:\n\n"); foreach (string s in q) { mydate dcheck = setanydate(s,kf,false); } string[] qs = { "2 Feb 2020", "6 Feb 2020", "20 Feb 2020", "25 Feb 2020", "1 Mar 2020", "3 Mar 2020", "30 Mar 2020", "2 Apr 2020", "18 Apr 2020", }; string[] ds = {"y","m","d"}; bool certain = false; print("\nbatch test on a sequence, with correction:\n\n"); foreach (string s in qs) { mydate dcheck = setanydate(s,kf,false); if (dcheck.certainty == 3) { kf = dcheck.format; certain = true; print("\ncertain format is %s %s %s\n\n",ds[kf[0]],ds[kf[1]],ds[kf[2]]); break; } } if (certain) { foreach (string s in qs) { mydate dcheck = setanydate(s,kf,false); } } }
#+RESULTS:
Compilation succeeded - 2 warning(s) batch test on a random date formats: dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 10 is month (year is last, i am 2nd) dateparts[0] 9 is day (year & month are taken) original date string : 9/10/2023 year is 2023 month is 10 day is 9 certainty is 3 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 9 is month (year is last, i am 2nd) dateparts[0] 10 is day (year & month are taken) original date string : 10/9/2023 year is 2023 month is 9 day is 10 certainty is 3 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[0] 20 is day (n > 12 & year is taken) dateparts[1] 11 is month (year is last, i am 2nd) original date string : 20/11/2023 year is 2023 month is 11 day is 20 certainty is 3 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 20 is day (n > 12 & year is taken) dateparts[0] 11 is month (year + day are taken) original date string : 11/20/2023 year is 2023 month is 11 day is 20 certainty is 3 dateparts[0] 23 is year? (fallback) dateparts[1] 9 is month? (fallback) dateparts[2] 10 is day (year & month are taken) original date string : 23-9-10 year is 23 month is 9 day is 10 certainty is 1 dateparts[0] 9 is day? (fallback) dateparts[2] 23 is year? (fallback) dateparts[1] 10 is month (year + day are taken) original date string : 9 10 23 year is 23 month is 10 day is 9 certainty is 1 original date string : 230910 year is 23 month is 9 day is 10 original date string : 20231102 year is 2023 month is 11 day is 2 dateparts[0] 11 is day? (fallback) dateparts[1] 30 is year? (fallback) original date string : 11 30 23 year is 30 month is -1 day is 11 certainty is 0 dateparts[1] 9 is month (found string sep) thedate[1] (month) is 9 dateparts[0] 10 is day? (fallback) dateparts[2] 23 is year? (fallback) original date string : 10 sep 23 year is 23 month is 9 day is 10 certainty is 1 dateparts[1] 2 is month (found string feb) thedate[1] (month) is 2 dateparts[2] 2020 is year (4-num & > 1700 & < 2070) dateparts[0] 20 is day (n > 12 & year is taken) original date string : 20 Feb 2020 year is 2020 month is 2 day is 20 certainty is 3 dateparts[0] 9 is month (found string sep) thedate[1] (month) is 9 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 10 is day (year & month are taken) original date string : sep 10 2023 year is 2023 month is 9 day is 10 certainty is 3 dateparts[0] 9 is month (found string september) thedate[1] (month) is 9 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 10 is day (year & month are taken) original date string : september 10th, 2023 year is 2023 month is 9 day is 10 certainty is 3 dateparts[1] 9 is month (found string september) thedate[1] (month) is 9 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[0] 10 is day (year & month are taken) original date string : 10th of September, 2023 year is 2023 month is 9 day is 10 certainty is 3 dateparts[0] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 10 is day (year is @0, i am last) dateparts[2] 9 is month (year + day are taken) original date string : 2023-10-09T22:07:23+00:00 year is 2023 month is 9 day is 10 certainty is 3 dateparts[0] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 10 is day (year is @0, i am last) dateparts[2] 9 is month (year + day are taken) original date string : 2023-10-09 22:07:23 +00:00 year is 2023 month is 9 day is 10 certainty is 3 dateparts[0] 10 is month (found string october) thedate[1] (month) is 10 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 10 is day (year & month are taken) original date string : Tuesday October 10th, 2023 year is 2023 month is 10 day is 10 certainty is 3 dateparts[0] 10 is month (found string october) thedate[1] (month) is 10 dateparts[2] 2023 is year (4-num & > 1700 & < 2070) dateparts[1] 10 is day (year & month are taken) original date string : 9:50AM Tuesday October 10th, 2023 year is 2023 month is 10 day is 10 certainty is 3 batch test on a sequence, with correction: dateparts[1] 2 is month (found string feb) thedate[1] (month) is 2 dateparts[2] 2020 is year (4-num & > 1700 & < 2070) dateparts[0] 2 is day (year & month are taken) original date string : 2 Feb 2020 year is 2020 month is 2 day is 2 certainty is 3 certain format is d m y original date string : 2 Feb 2020 year is 2020 month is 2 day is 2 original date string : 6 Feb 2020 year is 2020 month is 2 day is 6 original date string : 20 Feb 2020 year is 2020 month is 2 day is 20 original date string : 25 Feb 2020 year is 2020 month is 2 day is 25 original date string : 1 Mar 2020 year is 2020 month is 3 day is 1 original date string : 3 Mar 2020 year is 2020 month is 3 day is 3 original date string : 30 Mar 2020 year is 2020 month is 3 day is 30 original date string : 2 Apr 2020 year is 2020 month is 4 day is 2 original date string : 18 Apr 2020 year is 2020 month is 4 day is 18
DSVALIGN:vala:

Align columns of a dsv string to acommodate their content.

// expand a dsv string to make it more readable // by c.p.brown, 2025 string expanddsv (string[] dat, int dsz, string d) { string o = ""; int[] cw = new int[dsz]; int rowcount = dat.length / dsz; string dlm = (" " + d + " "); for (int c = 0; c < dsz; c++) { int maxc = 0; for (int r = 0; r < rowcount; r++) { maxc = int.max(maxc,dat[(dsz*r)+c].strip().char_count()); } cw[c] = maxc; } for (int r = 0; r < rowcount; r++) { for (int c = 0; c < dsz; c++) { string cel = dat[(dsz*r)+c].strip(); int cc = cel.char_count(); if (c == (dsz - 1)) { o = "%s%s".printf(o,cel); } else { string spc = ""; if ((cw[c]-cc) > 0) { spc = "%*s".printf((cw[c]-cc)," "); } o = "%s%s%s%s".printf(o,cel,spc,dlm); } } o = "%s\n".printf(o); } return o; } void main() { string s = "column_0;column_1;column_2;column_3\none;twelve thousand eight hundred and seven;ten;thirty\nseventeen hundred;sixty;ninety eight;eleven\nthirty two;five;three thousand and four;fourty three"; print("\noriginal csv:\n%s\n",s); string[] dat = {}; string[] rows = s.split("\n"); int dsz = 1; for (int i = 0; i < rows.length; i++) { string[] cols = rows[i].split(";"); dsz = int.max(dsz,cols.length); for (int c = 0; c < cols.length; c++) { dat += cols[c]; } } print("\nexpanded csv:\n%s\n",expanddsv(dat,dsz,";")); }
#+RESULTS:
original csv: column_0;column_1;column_2;column_3 one;twelve thousand eight hundred and seven;ten;thirty seventeen hundred;sixty;ninety eight;eleven thirty two;five;three thousand and four;fourty three expanded csv: column_0 ; column_1 ; column_2 ; column_3 one ; twelve thousand eight hundred and seven ; ten ; thirty seventeen hundred ; sixty ; ninety eight ; eleven thirty two ; five ; three thousand and four ; fourty three
DUHWADE:vala:

Get parent of relative path

// get parent relative path of s // by c.p.brown 2024 string getrelativedirparent (string s) { string filepath = s; string pfd = "../"; if (filepath.has_suffix("/") == false) { filepath = filepath + "/"; } bool eatit = false; if (strcmp(filepath,"./") == 0) { pfd = "../"; } else { if (filepath.has_prefix("./")) { eatit = true;; } else { string nbk = filepath.replace("../","").strip(); if (nbk.char_count() == 0) { pfd = "../" + filepath; } else { if (filepath.has_prefix("../")) { eatit = true; } } } } if (eatit) { int ldl = filepath.last_index_of("/"); if (ldl != -1) { pfd = filepath.substring(0,ldl); ldl = pfd.last_index_of("/"); if (ldl != -1) { pfd = pfd.substring(0,ldl); pfd = pfd + "/"; } else { print("error: cant find 2nd last \"/\" in %s\n",filepath); } } else { print("error: cant find last \"/\" in %s\n",filepath); } } return pfd; } void main() { string[] s = { "./", "./a/", "./a/b/", "./a/b/c/", "../", "../../", "../../../a/", "../../../a/b/", ".", "./a", "./a/b", "./a/b/c", "..", "../..", "../../../a", "../../../a/b" }; for (int i = 0; i < s.length; i++) { string pfd = getrelativedirparent(s[i]); print("parent dir of %*s is %s\n",14,s[i],pfd); } }
#+RESULTS:
parent dir of ./ is ../ parent dir of ./a/ is ./ parent dir of ./a/b/ is ./a/ parent dir of ./a/b/c/ is ./a/b/ parent dir of ../ is ../../ parent dir of ../../ is ../../../ parent dir of ../../../a/ is ../../../ parent dir of ../../../a/b/ is ../../../a/ parent dir of . is ../ parent dir of ./a is ./ parent dir of ./a/b is ./a/ parent dir of ./a/b/c is ./a/b/ parent dir of .. is ../../ parent dir of ../.. is ../../../ parent dir of ../../../a is ../../../ parent dir of ../../../a/b is ../../../a/
FAKESCROLL:vala:

Math for fake scrolling data where there's a variable number of lines per row.
Has to work with scaling.
Inputs are:
  • line height (scaled)
  • list of lines
  • list of rows
  • list of row indices per line
  • list of lines per row
  • position offset (from user, eg scroll offset)
Coords are top-left, iverted y (down is +y, as per cairo context)

This extends linesperrow below

double dubmod (double l, double r) { if (l >= 0) { return (l % r); } if (l >= -r) { return (l + r); } return ((l % r) + r) % r; } void main() { int[] positions = {0,-10,-20,-30,-40,-50,-60,-70,-80,-90,-100,-110,-120,-130,-140,-150,-160,-170}; int[] rows = {0,1,2}; int[] rowlines = {1,1,3,1}; int[] linerows = {0,1,2,2,2,3}; double lineheight = 30; // simulate scroll from 0 to -17 double rdwn = 0.0; int oldrow = 0; foreach (int posy in positions) { int lineoffset = (int) (posy.abs() / lineheight); int rowoffset = linerows[lineoffset]; if (rowoffset != oldrow) { oldrow = rowoffset; rdwn = posy; } double rowsizey = rowlines[rowoffset] * lineheight; double vsy = (rdwn - posy); double fakeposy = dubmod(posy.abs(),rowsizey); print("pos %05.1f : line %d row %d : size y is %.1f : fakescroll is %.1f : vsy is %.1f\n",posy,lineoffset,rowoffset,lineheight,fakeposy,vsy); } }
#+RESULTS:
Compilation succeeded - 1 warning(s) pos 000.0 : line 0 row 0 : size y is 30.0 : fakescroll is 0.0 : vsy is 0.0 pos -10.0 : line 0 row 0 : size y is 30.0 : fakescroll is 10.0 : vsy is 10.0 pos -20.0 : line 0 row 0 : size y is 30.0 : fakescroll is 20.0 : vsy is 20.0 pos -30.0 : line 1 row 1 : size y is 30.0 : fakescroll is 0.0 : vsy is 0.0 pos -40.0 : line 1 row 1 : size y is 30.0 : fakescroll is 10.0 : vsy is 10.0 pos -50.0 : line 1 row 1 : size y is 30.0 : fakescroll is 20.0 : vsy is 20.0 pos -60.0 : line 2 row 2 : size y is 30.0 : fakescroll is 60.0 : vsy is 0.0 pos -70.0 : line 2 row 2 : size y is 30.0 : fakescroll is 70.0 : vsy is 10.0 pos -80.0 : line 2 row 2 : size y is 30.0 : fakescroll is 80.0 : vsy is 20.0 pos -90.0 : line 3 row 2 : size y is 30.0 : fakescroll is 0.0 : vsy is 30.0 pos -100.0 : line 3 row 2 : size y is 30.0 : fakescroll is 10.0 : vsy is 40.0 pos -110.0 : line 3 row 2 : size y is 30.0 : fakescroll is 20.0 : vsy is 50.0 pos -120.0 : line 4 row 2 : size y is 30.0 : fakescroll is 30.0 : vsy is 60.0 pos -130.0 : line 4 row 2 : size y is 30.0 : fakescroll is 40.0 : vsy is 70.0 pos -140.0 : line 4 row 2 : size y is 30.0 : fakescroll is 50.0 : vsy is 80.0 pos -150.0 : line 5 row 3 : size y is 30.0 : fakescroll is 0.0 : vsy is 0.0 pos -160.0 : line 5 row 3 : size y is 30.0 : fakescroll is 10.0 : vsy is 10.0 pos -170.0 : line 5 row 3 : size y is 30.0 : fakescroll is 20.0 : vsy is 20.0
FILESEEK:vala:

Read a file as 100-char lines
Use of StringBuilder.append((string) buf) was found to prevent both buffer overflow and incorrect rendering of unicode.

void main() { GLib.Intl.setlocale(ALL,""); print(","); char[] charline = new char[100]; GLib.FileStream fstr = GLib.FileStream.open("./cpbrown_notes.org","r"); for (int r = 0; r < 10; r++) { StringBuilder sbline = new StringBuilder(""); if (fstr.gets(charline) != null) { sbline.append((string) charline); sbline.replace("\n","\n,",-1); print(sbline.str); } else { break; } } }
#+RESULTS:
,# -*- mode:org; org-confirm-babel-evaluate: nil; org-todo-keyword-faces: (("[0_TODO]" . "orange") ("[1_IP..]" . "yellow") ("[2_FIX.]" . "red") ("[3_WAIT]" . "blue") ("[4_NOPE]" . "black") ("[5_DONE]" . "green")); -*- ,#+STARTUP: indent overview align ,#+OPTIONS: toc:nil num:nil title:nil \n:t author:nil ::nil H:1 f:nil ,#+TITLE: the work ,#+AUTHOR: c.p.brown ,#+TODO: [0_TODO] [1_IP..] [2_FIX.] [3_WAIT] [4_NOPE] [5_DONE] ,#+SUBTITLE: ,#+NOTHING: 零 goes here ,
FILEBYIDX:wip::vala:

Re-arrange a string array using an int array of indices.
Requres more testing for edge cases...

string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } public void filebyindices (ref string[] dat, ref int[] dxs) { int i = 0; int l = dxs.length; int li = l - 1; for (int n = 0; n < l; n++) { if ((dxs[dxs[i]] == -1) || (dxs[i] == i)) { dxs[i] = -1; int h = l / 2; int d = -1; int q = 0; if (i < h) { d = 1; } while (dxs[i] == -1) { if (q > l) { break; } i = ((i + d) % l); q += 1; } } print("i = %d, li = %d\n",i,li); print("ROUND %d\n| I |",n); for (int j = 0; j < l; j++) { if (i == j) { print(" %02d |",j); } else { print(" |"); } } print("\n| X |"); for (int j = 0; j < l; j++) { print(" %02d |",dxs[j]); } print("\n| C |"); for (int j = 0; j < l; j++) { print(" %s |",dat[j]); } print("\n| |"); for (int j = 0; j < l; j++) { if (i == j) { print(" ^^ "); } else { if (dxs[i] == j) { if (j == li) { print(" ^^ "); } else { print(" ^^ "); } } else { if (j == li) { print(" "); } else { print(" "); } } } } print("|\n| C |"); if ((dxs[dxs[i]] >= 0) && (dxs[i] != i)) { if (n < (li - 1)) { string t = dat[i]; dat[i] = dat[dxs[i]]; dat[dxs[i]] = t; int p = i; i = dxs[i]; dxs[p] = -1; } else { for (int j = 0; j < l; j++) { print(" %s |",dat[j]); } print("\n| X |"); for (int j = 0; j < l; j++) { print(" %02d |",dxs[j]); } print("\n\n"); //print("n %d >= li %d, breaking\n",n,li); break; } } else { for (int j = 0; j < l; j++) { print(" %s |",dat[j]); } print("\n| X |"); for (int j = 0; j < l; j++) { print(" %02d |",dxs[j]); } print("\n\n"); if (dxs[i] == i) { continue; } if (dxs[dxs[i]] < 0) { break; } } for (int j = 0; j < l; j++) { print(" %s |",dat[j]); } print("\n| X |"); for (int j = 0; j < l; j++) { print(" %02d |",dxs[j]); } print("\n\n"); } } void main() { GLib.Intl.setlocale(ALL,""); string[] s = { "JJ", "AA", "DD", "SS", "PP", "DD" }; int[] k = { 1,5,2,5,4,3 }; print("DATA\n"); for (int i = 0; i < s.length; i++) { print("s[%d] is %s\n",i,s[i]); } print("\nTEST\n\n"); int64 tts = GLib.get_monotonic_time(); filebyindices(ref s,ref k); int64 tte = GLib.get_monotonic_time(); print("filebyindices toolk %s\n",printusecs(tte-tts)); print("\nDATA\n"); for (int i = 0; i < s.length; i++) { print("s[%d] is %s\n",i,s[i]); } }
#+RESULTS:
DATA s[0] is JJ s[1] is AA s[2] is DD s[3] is SS s[4] is PP s[5] is DD TEST i = 0, li = 5 ROUND 0 | I | 00 | | | | | | | X | 01 | 05 | 02 | 05 | 04 | 03 | | C | JJ | AA | DD | SS | PP | DD | | | ^^ ^^ | | | | | | | C | AA | JJ | DD | SS | PP | DD | | X | -1 | 05 | 02 | 05 | 04 | 03 | i = 1, li = 5 ROUND 1 | I | | 01 | | | | | | X | -1 | 05 | 02 | 05 | 04 | 03 | | C | AA | JJ | DD | SS | PP | DD | | | ^^ ^^ | | | | | | | C | AA | DD | DD | SS | PP | JJ | | X | -1 | -1 | 02 | 05 | 04 | 03 | i = 5, li = 5 ROUND 2 | I | | | | | | 05 | | X | -1 | -1 | 02 | 05 | 04 | 03 | | C | AA | DD | DD | SS | PP | JJ | | | ^^ ^^ | | | | | | | C | AA | DD | DD | JJ | PP | SS | | X | -1 | -1 | 02 | 05 | 04 | -1 | i = 2, li = 5 ROUND 3 | I | | | 02 | | | | | X | -1 | -1 | 02 | -1 | 04 | -1 | | C | AA | DD | DD | JJ | PP | SS | | | ^^ | | | | | | | C | AA | DD | DD | JJ | PP | SS | | X | -1 | -1 | 02 | -1 | 04 | -1 | i = 4, li = 5 ROUND 4 | I | | | | | 04 | | | X | -1 | -1 | -1 | -1 | 04 | -1 | | C | AA | DD | DD | JJ | PP | SS | | | ^^ | | | | | | | C | AA | DD | DD | JJ | PP | SS | | X | -1 | -1 | -1 | -1 | 04 | -1 | i = -1, li = 5 ROUND 5 | I | | | | | | | | X | -1 | -1 | -1 | -1 | -1 | -1 | | C | AA | DD | DD | JJ | PP | SS | | | ^^ | | | | | | | C | AA | DD | DD | JJ | PP | SS | | X | -1 | -1 | -1 | -1 | -1 | -1 | filebyindices toolk 198 μs DATA s[0] is AA s[1] is DD s[2] is DD s[3] is JJ s[4] is PP s[5] is SS
FOLD:vala:
tokenize_and_fold vs string._delimit & split...

_delimit pros
  • much faster, even with cleanup
  • preserves case
  • can choose exactly what to use as delimiters

_delimit cons
  • won't work with multi-character delimiters - as with tokenize, to be fair
  • will insert sequential delimiters, resulting in empty array items when split
  • adds an extra empty array item at the end if the last char is a delimiter
  • removing empties requires onerous cleanup work & testing,
  • or handling emties every time the array is accessed

// tokenize_and_fold vs _delimit & split // by c.p.brown 2024 void main() { string[] o; string s = "1,2;3\"45\"\n6-78\'AB.C_DE(F)$GH{I\nJK}AB101"; int64 tts = GLib.get_monotonic_time(); string[] tkn = s.tokenize_and_fold("en_US",out o); int64 ttl = GLib.get_monotonic_time(); print("string.tokenize_and_fold is {\n"); for (int i = 0; i < tkn.length; i++) { print("\"" + tkn[i] + "\"\n"); } print("}\n"); print("tokenize_and_fold took %.2f microseconds\n\n",((double) (ttl - tts))); string q = "1,2;3\"45\"\n6-78\'AB.C_DE(F)$GH{I\nJK}AB101"; string d = ",;\"\';._-$(){}\n"; char npdc = '\x1F'; string npd = "%c".printf(npdc); tts = GLib.get_monotonic_time(); q._delimit(d,npdc); string[] w = q.split(npd); int[] rr = {}; for (int i = 0; i < w.length; i++) { if ((w[i] == null) || (w[i].strip() == "") || (w[i].strip() == "\n")) { rr += i; } } if (rr.length > 1) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = rr.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = rr[i]; int j = i; if (j >= g) { bool c = (rr[j - g] < t); while (c) { rr[j] = rr[j - g]; j -= g; if (j < g) { break; } c = (rr[j - g] < t); } } rr[j] = t; } } n = 0; int a = rr[rr.length - 1]; int j = rr.length; int f = 0; for (int i = a; i < (w.length - j); i++) { int b = (i + f); while (b in rr) { f += 1; b += 1; rr.resize(rr.length - 1); } w[i] = w[i + f]; } w.resize(w.length - j); } else { if (rr.length == 1) { for (int i = rr[0]; i < w.length; i++) { if ((i + 1) < w.length) { w[i] = w[i + 1]; } } w.resize(w.length - 1); } } ttl = GLib.get_monotonic_time(); print("string._delimit and split is {\n"); for (int i = 0; i < w.length; i++) { print("\"" + w[i] + "\"\n"); } print("}\n"); print("_delimit and split took %.2f microseconds\n\n",((double) (ttl - tts))); }
#+RESULTS:
string.tokenize_and_fold is { "1" "2" "3" "45" "6" "78" "ab" "c" "de" "f" "gh" "i" "jk" "ab101" } tokenize_and_fold took 16.00 microseconds string._delimit and split is { "1" "2" "3" "45" "6" "78" "AB" "C" "DE" "F" "GH" "I" "JK" "AB101" } _delimit and split took 5.00 microseconds
FONTRESOURCE:busted::vala:

ugh I just want to load a font and have widgets use it...


int main(string[] args) { Gtk.Application myapp = new Gtk.Application("com.mytest.mytest",GLib.ApplicationFlags.DEFAULT_FLAGS); myapp.activate.connect (() => { //Pango.FontMap fc = Pango.FontMap.get_default(); Gtk.ApplicationWindow mywin = new Gtk.ApplicationWindow(myapp); Gtk.Label mylabel = new Gtk.Label("is this Mirchroma?"); //Pango.Context pc = mylabel.get_pango_context(); //Pango.FontMap fc = mylabel.get_pango_context().get_font_map(); Pango.FontDescription fd = Pango.FontDescription.from_string("Michroma-Regular"); mylabel.get_pango_context().get_font_map().load_font(mylabel.get_pango_context(),fd); //mylabel.set_pango_context(pc); //mylabel.get_style_context().add_provider(pc,Gtk.STYLE_PROVIDER_PRIORITY_APPLICATION); mywin.set_child(mylabel); mywin.default_width = 120; mywin.default_height = 70; mywin.present(); }); return myapp.run(args); }
#+RESULTS:

FORECAST:wip::vala:

Complex recurrence, revisited.

This revision intends to cover the following scenarios not handled previously:
  • Every nth day (or weekday) of a single month.
  • Every nth and nth day in alternation, either per-month or absolute, eg:
    • The 2nd and 3rd day of March (2,3).
    • The 2nd and 3rd day of March, from the 5th (7,8).
    • Every 2nd then 3rd day of March (2,5,7,10,12,15,17,20,22,25,27,30).
    • Every 2nd then 5th day of March, from the 10th (12,17,19,24,26).
    • The 2nd and 3rd Tuesday of March 2025 (11,18).
    • The 2nd and 3rd Tuesday of April 2025, from the 12th (22,29).
    • Every 2nd then 3rd Tuesday of April 2025 (8,29).
    • Every 90 and 91 days from July 1st 2025.
  • Every nth and nth month in alternation, eg:
    • The 2nd and 10th month of every year (2,10).
    • The 2nd and 5th month, from the 2nd month, of every year (4,7).
    • Every 2nd then 3rd month of every year (2,5,7,10,12).
    • Every 2nd then 3rd month, from the 3rd month, of every year (5,8,10).
    • Every 4th month from March 2025.
  • Every nth year.

// build complex recurrance rules using english // by c.p.brown, 2025 // // {1,10,0,8,0, 0,0,0,0,11, 0,0,0,2025} // | | | | | | // +--------+ +--------+ +--------+ // every|the of|of every of|of every // nth nth nth // then nth then nth this|year // day|weekd this|month this|from year // 0|from day this|from month // // TODO: // - return a Date[] of two dates: recurring-date and offset-date, // to report workday dates without affecting the recurrance // - resolve ambiguity: // "every nth day from the nth day of [month] from [none ] ..." // "every nth day from the nth day of [none ] from [month] ..." // - use the above to allow: // count absolute days from a dmy // count days per month, from a day of that month // - tests: // every 10th day from the 5th of (only) november of (only) 2025 // {1,10,0,0,5,0,0,0,2,0,0,0,2025,0} // = 15,25 // every 10th day from the 5th of november of (only) 2025 // {1,10,0,0,5,0,0,0,0,2,0,0,2025,0} // = 15,25,5,15,25 // every 10th day from the 5th of december of 2025 // {1,10,0,0,5,0,0,0,0,2,0,0,0,2025} // = 15,25,4,14,24... // the weekday on or before every 10th day of (only) december of (only) 2025 // {1,10,0,12,0,0,0,0,2,0,0,0,2025,0} // = 10,19,30 // the weekday on or before every 10th day of november of (only) 2025 // {1,10,0,8,0, 0,0,0,11,0, 0,0,2025,0} // = 10,20,28,10,19,30 // the weekday on or before every 10th day from november from 2025 // {1,10,0,8,0, 0,0,0,0,11, 0,0,0,2025} // = 10,20,28,10,19,30 int imod (int l, int r) { if (l >= 0) { return (l % r); } if (l >= -r) { return (l + r); } return ((l % r) + r) % r; } double dmod (double l, double r) { if (l >= 0) { return (l % r); } if (l >= -r) { return (l + r); } return ((l % r) + r) % r; } bool isly(int y) { // (year % 100 == 0) checks if the year is at the beginning of a century // (year % 400 == 0) checks if the above, if true, is every 400th year, which is a leap-year // (year % 4 == 0) otherwise, this checks if its every 4th year, which is a leap-year if ((y % 100) == 0 ) { return ((y % 400) == 0); } return ((y % 4) == 0); } int iwkd (DateWeekday wd) { if (wd == MONDAY) { return 1; } if (wd == TUESDAY) { return 2; } if (wd == WEDNESDAY) { return 3; } if (wd == THURSDAY) { return 4; } if (wd == FRIDAY) { return 5; } if (wd == SATURDAY) { return 6; } if (wd == SUNDAY) { return 7; } if (wd == BAD_WEEKDAY) { return 0; } return 0; } string printintarray (int[] s, int f) { if (s.length == 0) { return "(empty array)"; } string[] ii = {"", "\"", "{", "[", "[", "| ", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%d".printf(s[i]) + dd[f]); } o = (o + "%d".printf(s[s.length - 1]) + oo[f]); return o; } int lymd (int m, int y) { if (m == 2) { if ((y % 100) == 0) { if ((y % 400) == 0) { return 29; } } else { if ((y % 4) == 0) { return 29; } } return 28; } else { int[] lastdays = {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}; return lastdays[m]; } return -1; } string whiggle (int n, int p, int e) { if ((e == 1) && (n == 1)) { return "single"; } int l = n % 10; if (l == 0) { return "%0*dth".printf(p,n); } int ll = n % 100; if ((ll > 10) && (ll < 20)) { return "%0*dth".printf(p,n); } if (l < 4) { string[] w = {"th","st","nd","rd"}; return "%0*d%s".printf(p,n,w[l]); } return "%0*dth".printf(p,n); } int getdayoffsetofweekday(int d, int f, int w, int i, int m, int y, bool c) { GLib.Date j = Date(); int fd = int.max(d,f); //print("getdayoffsetofweekday fromday is %d max(previouisday %d,fromday %d)\n",fd,d,f); j.set_dmy((DateDay) fd, m,(DateYear) y); int jw = (int) j.get_weekday(); int dayoffset = (w - jw); //print("getdayoffsetofweekday weekday of %d-%d-%d is %d\n",y,m,fd,jw); //print("getdayoffsetofweekday weekday offset is %d\n",dayoffset); int nth = i; //print("getdayoffsetofweekday interval is %d\n",nth); //if (fd > d) { fd = f; } if (dayoffset == 0) { dayoffset = fd; } else { //dayoffset += 1; if (dayoffset < 0) { dayoffset = (fd + dayoffset); } else { dayoffset = ((fd + dayoffset) - (nth - 7)); } } //print("getdayoffsetofweekday returnts %d\n",dayoffset); return dayoffset; } Date getnextdate(Date s, Date n, Date b, int[] r, int q) { int[] predictions = {}; GLib.Date a = Date(); a.set_dmy((DateDay) s.get_day(), s.get_month(), (DateYear) s.get_year()); GLib.Date nd = Date(); //a.set_dmy((DateDay) 1, 1, (DateYear) 1989); int nty = (int) n.get_year(); int ntm = (int) n.get_month(); int ntd = (int) n.get_day(); int dd = (int) a.get_day(); int mm = (int) a.get_month(); int yy = (int) a.get_year(); int ld = (int) b.get_day(); int lm = (int) b.get_month(); int ly = (int) b.get_year(); // 00 whatday..... : the, every // 01 nthday...... : 1st, 2nd, 3rd, etc // 02 andnthday... : "", and 2nd, and 3rd, and 4th, etc. (when used with 'every', it alternates) // 03 weekday..... : day, monday, tuesday, wednesday, thursday, friday, saturday, sunday, weekday on or before the, weekday on or after the, weekday closest to the, last // 04 fromday..... : "", from the 1st day, from the 2nd day, from the 3rd day, ... from the last day // 05 whatmonth... : of, of every // 06 nthmonth.... : "", 1st, 2nd, 3rd, etc. // 07 andnthmonth. : "", and 1st, and 2nd, and 3rd, etc. // 08 themonth.... : "", january, february, march, etc. // 09 frommonth... : "", from january, from february, from march, etc. // 10 whatyear.... : of, of every // 11 nthyear..... : "", year, 2nd year, 3rd year, etc. // 12 theyear..... : "", 1990 ... 2026 // 13 fromyear.... : "", 1990 ... 2026 string[] lomo = {"null","January","February","March","April","May","June","July","August","September","October","December"}; string[] lowd = {"null","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday","Sunday"}; int[,] wwkd = {{0,0,0,0,0,0,0,0},{0,0,0,0,0,0,-1,-2},{0,0,0,0,0,0,2,1},{0,0,0,0,0,0,-1,1}}; int whatday = r[0]; // "the" or "every" int nthday = r[1]; int andnthday = r[2]; int weekd = r[3]; int fromday = r[4]; // "from the" int whatmonth = r[5]; // "of the" or "of every" int nthmonth = r[6]; int andnthmonth = r[7]; int themonth = r[8]; int frommonth = r[9]; // "from the" int whatyear = r[10]; // "of the" or "of every" int nthyear = r[11]; int theyear = r[12]; // "from" int fromyear = int.max(1990,r[13]); if (r[13] == 0) { fromyear = nty; } //fromyear = fromyear - 1990; //print("fromyear is %d, at index %d\n",fromyear,(fromyear - 1990)); // offsets are used at the begining of each month/year if we're counting per month/year // otherwise they are used to set an initial start date if we're day counting from a start date int dayoffset = 0; int monthoffset = 0; int yearoffset = 0; int[] dayintervals = {1,0}; int[] monthintervals = {1,0}; int yearinterval = 1; // if true, the following determine if we need to recalculate offsets per year/month bool countpermonth = false; bool countperyear = false; if ((nthyear > 0) || (whatyear == 1)) { countperyear = true; } if ((nthmonth > 0) ||(whatmonth == 1)) { countpermonth = true; } // don't calc from-dmy if there's no "every" or "nth" repetition bool oneday = false; bool onemonth = false; bool oneyear = false; if ((whatyear == 0) && (nthyear == 0)) { fromyear = 0; oneyear = true; } if ((whatmonth == 0) && (nthmonth == 0)) { frommonth = 0; } if ((whatmonth == 0) && oneyear) { onemonth = true; } if ((whatday == 0) && (nthday == 0)) { fromday = 0; } if ((whatday == 0) && onemonth) { oneday = true; } if ((whatmonth > 0) && (nthmonth == 0)) { nthmonth = 1; } // if its a single specific date, check and return it if (oneday && onemonth && oneyear) { if (q > 0) { return nd; } if (theyear < 1990) { a.set_year((DateYear) nty); yy = (int) a.get_year(); } else { a.set_year((DateYear) theyear); yy = (int) a.get_year(); } yy = (int) a.get_year(); if (themonth < 1) { a.set_month(ntm); mm = (int) a.get_month(); } else { a.set_month(themonth); mm = (int) a.get_month(); } if (nthday == 0) { a.set_day((DateDay) int.min(ntd,lymd(mm,yy))); dd = (int) a.get_day(); } else { //print("getnextdate (single date) nth is %d, fromday is %d\n",nthday,fromday); int nth = nthday; dayoffset = int.max((fromday - 1),0); if ((q & 1) != 0) { nth = andnthday; } if (weekd > 0) { nth = nthday * 7; //print("getnextdate (single date) nthday is %d, weekday is %d, fromday is %d, month is %d, year is %d, calculating offset...\n",nthday,weekd,fromday, mm, yy); dayoffset = getdayoffsetofweekday(dd, fromday, weekd, nth, mm, yy, countpermonth); //print("getnextdate (single date) dayoffset is %d, + nth %d * 7 (%d) days is %d\n",dayoffset,nthday,nth,(dayoffset + nth)); } a.set_day((DateDay) (dayoffset + nth)); dd = (int) a.get_day(); if ((dayoffset + nth) > lymd(mm,yy)) { print("getnextdate (single date) returns out-of-range date\n"); return nd; } } // return an invalid date, checked in the calling code and not collected. print("getnextdate (single date) returns %d-%d-%d\n",yy,mm,dd); return a; } // if its an nth-day count from a specific dmy, check and return it if (onemonth && oneyear) { // if the forecast iteration is 0, calc the initial offset if (q == 0) { if (theyear > 0) { a.set_year((DateYear) theyear); } if (themonth > 0) { a.set_month(themonth); } int nth = nthday; //if (andnthday > 0) { if ((q & 1) == 0) { nth = andnthday; } } dayoffset = int.max((fromday - 1),0); if ((weekd > 0) && (weekd < 8)) { nth = nth * 7; //print("getnextdate (daycounter) nth is %d, weekday is %d, fromday is %d, calculating offset...\n",nthday,weekd,fromday); dayoffset = getdayoffsetofweekday(dd, fromday, weekd, nth, mm, yy, countpermonth); //print("getnextdate (daycounter) dayoffset is %d, + nth %d * 7 (%d) days is %d\n",dayoffset,nthday,nth,(dayoffset + nth)); } else { print("[%d] getnextdate (daycounter) dayoffset %d, + nth %d days is %d (%d-%d-%d + %d)\n",q,dayoffset,nth,(dayoffset + nth),yy,mm,dd,(dayoffset + nth)); } if (dayoffset > lymd(mm,yy)) { print("[%d] getnextdate (daycounter) returns out-of-range date\n",q); return nd; } else { a.set_day((DateDay) dayoffset); dd = (int) a.get_day(); } if (weekd > 7) { // get work-weekday on-or-before (dayoffset + nth) GLib.Date j = new GLib.Date(); j.set_dmy((DateDay) dd, mm, (DateYear) yy); j.add_days(nth); int nwd = j.get_weekday(); print("[%d] getnextdate (daycounter) %d-%d-%d weekday is %d \n",q,((int) j.get_year()),((int) j.get_month()),((int) j.get_day()),nwd); if (nwd > 5) { int nwe = int.max(0,(weekd - 7)); nth = nth + wwkd[nwe,nwd]; } } a.add_days(nth); dd = (int) a.get_day(); } else { int nth = nthday; if (andnthday > 0) { if ((q & 1) != 0) { nth = andnthday; } } if (nth == 0) { nth = 1; } if ((weekd > 0) && (weekd < 8)) { nth = nth * 7; } else { if (weekd > 7) { GLib.Date j = new GLib.Date(); j.set_dmy((DateDay) dd, mm, (DateYear) yy); j.add_days(nth); int nwd = j.get_weekday(); print("[%d] getnextdate (daycounter) %d-%d-%d weekday is %d \n",q,((int) j.get_year()),((int) j.get_month()),((int) j.get_day()),nwd); if (nwd > 5) { int nwe = int.max(0,(weekd - 7)); nth = nth + wwkd[nwe,nwd]; } } } a.add_days(nth); } dd = (int) a.get_day(); mm = (int) a.get_month(); yy = (int) a.get_year(); print("[%d] getnextdate (daycounter) returns %d-%d-%d\n",q,yy,mm,dd); return a; } // get next day using fromday + nthday int nth = nthday; if (q == 0) { dd = 0; } dayoffset = int.max(dd,int.max(0,(fromday - 1))); if (andnthday > 0) { if ((q & 1) != 0) { nth = andnthday; } } print("[%d] getnextdate (init) previous day %d, from day %d, weekday %d, nthday %d, month %d, year %d\n",q,dd,fromday,weekd,nthday,mm,yy); if ((weekd > 0) && (weekd < 8)) { nth = (nth * 7); dayoffset = getdayoffsetofweekday(dd, fromday, weekd, nth, mm, yy, countpermonth); } else { if (weekd > 7) { // get work-weekday on-or-before (dayoffset + nth) GLib.Date j = new GLib.Date(); j.set_dmy((DateDay) (dayoffset + nth), mm, (DateYear) yy); int nwd = j.get_weekday(); print("[%d] getnextdate (init) day %d weekday is %d \n",q,(dayoffset + nth),nwd); if (nwd > 5) { int nwe = int.max(0,(weekd - 7)); nth = nth + wwkd[nwe,nwd]; } } } if (countpermonth) { print("[%d] getnextdate (countpermonth) dayoffset %d + nth %d is %d\n",q,dayoffset,nth,(dayoffset + nth)); if ((nth + dayoffset) > lymd(mm,yy)) { if (countperyear) { if ((mm + nthmonth) > 12) { // per month per year, rolling over to +nth month, +nth year if (nthyear > 0) { a.add_years(nthyear); yy = (int) a.get_year(); a.set_month(frommonth); mm = (int) a.get_month(); a.set_day((DateDay) 1); dd = (int) a.get_day(); dayoffset = fromday; if ((weekd > 0) && (weekd < 8)) { dayoffset = getdayoffsetofweekday(dd, fromday, weekd, nthday, mm, yy,countpermonth); } a.set_day((DateDay) (dayoffset + nth)); dd = (int) a.get_day(); if ((dayoffset + nth) > lymd(mm,yy)) { //print("getnextdate (per month rollover per year rollover) dayoffset %d + nth %d is invalid\n",dayoffset,nth); return nd; } else { //print("getnextdate (per month rollover per year rollover) returns %d-%d-%d\n",yy,mm,dd); return a; } } else { //print("getnextdate (per month rollover per year rollover) reached end of a single year, returning invalid date\n"); return nd; } } } // per month, rolling over to +nth month a.add_months(nthmonth); mm = (int) a.get_month(); a.set_day((DateDay) 1); dd = (int) a.get_day(); print("[%d] getnextdate (per month rollover) incremented +%d month is %d-%d-%d\n",q,nthmonth,yy,mm,dd); dayoffset = fromday; if ((weekd > 0) && (weekd < 8)) { dayoffset = getdayoffsetofweekday(dd, fromday, weekd, nth, mm, yy,countpermonth); print("[%d] getnextdate (per month rollover) new monthly offset is %d\n",q,dayoffset); } else { if (weekd > 7) { // get work-weekday on-or-before (dayoffset + nth) GLib.Date j = new GLib.Date(); j.set_dmy((DateDay) (dayoffset + nth), mm, (DateYear) yy); int nwd = j.get_weekday(); print("[%d] getnextdate (per month rollover) day %d weekday is %d \n",q,(dayoffset + nth),nwd); if (nwd > 5) { int nwe = int.max(0,(weekd - 7)); nth = nth + wwkd[nwe,nwd]; print("[%d] getnextdate (per month rollover) day %d weekday %d offset type is %d, offset is %d, nth is now %d\n",q,(dayoffset + nthday),nwd,nwe,wwkd[nwe,nwd],nth); } } } if ((dayoffset + nth) > lymd(mm,yy)) { print("[%d] getnextdate (per month rollover) dayoffset %d + nth %d is invalid\n",q,dayoffset,nth); return nd; } else { a.set_day((DateDay) (dayoffset + nth)); dd = (int) a.get_day(); print("[%d] getnextdate (per month rollover) returns %d-%d-%d\n\n",q,yy,mm,dd); return a; } } else { // per month a.set_day((DateDay) (dayoffset + nth)); dd = (int) a.get_day(); if ((dayoffset + nth) > lymd(mm,yy)) { print("getnextdate (per month) dayoffset %d + nth %d is invalid\n",dayoffset,nth); return nd; } else { print("[%d] getnextdate (per month) returns %d-%d-%d\n\n",q,yy,mm,dd); return a; } } } //print("getnextdate (per month no rollover) did not incrament from %d to %d+%d, returning invalid date\n",dd,dd,dayoffset); return nd; } void printrule (int[] r) { GLib.DateTime nt = new DateTime.now_local(); int ntd = nt.get_day_of_month(); int ntm = nt.get_month(); int nty = nt.get_year(); int whatday = r[0]; // "the" or "every" int nthday = r[1]; int andnthday = r[2]; int weekd = r[3]; int fromday = r[4]; // "from the" int whatmonth = r[5]; // "of the" or "of every" int nthmonth = r[6]; int andnthmonth = r[7]; int themonth = r[8]; int frommonth = r[9]; // "from the" int whatyear = r[10]; // "of the" or "of every" int nthyear = r[11]; int theyear = r[12]; // int fromyear = r[13]; //if (r[13] == 0) { fromyear = nty; } string[] whd = {"the","every"}; string[] nth = {}; string[] ath = {}; nth += ""; ath += ""; for (int i = 1; i < 366; i++) { nth += "%s".printf(whiggle(i,0,1)); ath += "then %s".printf(whiggle(i,0,1)); } string[] wkd = {"day","monday","tuesday","wednesday","thursday","friday","saturday","sunday","weekday on or before","weekday on or after","weekday closest to"}; string[] fdy = {"","from the 1st","from the 2nd","from the 3rd","from the 4th","from the 5th","from the 6th","from the 7th","from the 8th","from the 9th","from the 10th","from the 11th","from the 12th","from the 13th","from the 14th","from the 15th","from the 16th","from the 17th","from the 18th","from the 19th","from the 20th","from the 21st","from the 22nd","from the 23rd","from the 24th","from the 25th","from the 26th","from the 27th","from the 28th","from the 29th","from the 30th","from the 31st","from the last"}; string[] whm = {"of","of every"}; string[] mth = {"","1st","2nd","3rd","4th","5th","6th","7th","8th","9th","10th","11th","12th"}; string[] amh = {"","and 1st","and 2nd","and 3rd","and 4th","and 5th","and 6th","and 7th","and 8th","and 9th","and 10th","and 11th","and 12th"}; string[] thm = {"","january","february","march","april","may","june","july","august","september","october","november","december"}; string[] fmo = {"","from january","from february","from march","from april","from may","from june","from july","from august","from september","from october","from november","from december"}; string[] wye = {"of","of every"}; string[] yth = {"","year","2nd year", "3rd year", "4th year", "5th year", "6th year", "7th year", "8th year", "9th year", "decade"}; string[] thy = {}; thy += ""; for (int i = 1990; i < (nty + 2); i++) { thy += "%d".printf(i); } string[] fye = {}; fye += ""; for (int i = 1990; i < (nty + 2); i++) { fye += "%d".printf(i); } int theyearindex = 0; int fromyearindex = 0; if (theyear > 0) { theyearindex = theyear - 1989; } if (fromyear > 0) { fromyearindex = fromyear - 1989; } if ((themonth == 0) && (whatmonth == 0)) { thm[0] = "this month"; } if ((themonth == 0) && (whatmonth == 1)) { thm[0] = "month"; } if ((theyear == 0) && (whatyear == 0)) { thy[0] = "this year"; } print( "\"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\"\n", whd[whatday], nth[nthday], ath[andnthday], wkd[weekd], fdy[fromday], whm[whatmonth], mth[nthmonth], amh[andnthmonth], thm[themonth], fmo[frommonth], wye[whatyear], yth[nthyear], thy[theyearindex], fye[fromyearindex] ); } int[] forecast(int[] r) { int[] predictions = {}; // now GLib.DateTime nt = new DateTime.now_local(); int ntd = nt.get_day_of_month(); int ntm = nt.get_month(); int nty = nt.get_year(); GLib.Date n = Date(); n.set_dmy((DateDay) ntd, ntm, (DateYear) nty); // start counting from date a GLib.Date a = Date(); a.set_dmy((DateDay) 1, ntm, (DateYear) nty); //int mm = ((ntm - 2) % 12) + 1; //int yy = nty; //if (mm > ntm) { yy = nty - 1; a.set_year((DateYear) yy); } //a.set_month(mm); //a.set_day((DateDay) lymd(mm,yy)); // stop counting to date b GLib.Date b = Date(); b.set_dmy((DateDay) ntd,ntm,(DateYear) nty); b.add_years(1); for (int i = 0; i < 5; i++) { if (a.compare(b) > 0) { break; } //print("\nforecast iteration %d\n",i); a = getnextdate(a,n,b,r,i); if (((int) a.get_year()) > 1989) { if (a.compare(n) > 0) { predictions += (int) a.get_year(); predictions += (int) a.get_month(); predictions += (int) a.get_day(); } } else { print("no more dates\n"); break; } } return predictions; } void main () { // 00 whatday..... : the, every // 01 nthday...... : "", 1st, 2nd, 3rd, etc // 02 andnthday... : "", and 2nd, and 3rd, and 4th, etc. (when used with 'every', it alternates) // 03 weekday..... : day, monday, tuesday, wednesday, thursday, friday, saturday, sunday, weekday on or before the, weekday on or after the, weekday closest to the, last // 04 fromday..... : "", from the 1st day, from the 2nd day, from the 3rd day, ... from the last day // 05 whatmonth... : of, of every // 06 nthmonth.... : "", 1st, 2nd, 3rd, etc. // 07 andnthmonth. : "", then 1st, then 2nd, then 3rd, etc. // 08 themonth.... : "", january, february, march, etc. // 09 frommonth... : "", from january, from february, from march, etc. // 10 whatyear.... : of, of every // 11 nthyear..... : "", year, 2nd year, 3rd year, etc. // 12 theyear..... : "", 1990 ... 2026 <- limits are for ux reasons // 13 fromyear.... : "", 1990 ... 2026 int[] rule = {0,2,0,0,0,0,0,0,3,0,0,0,2026,0}; printrule(rule); int[] forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {0,2,0,2,0,0,0,0,0,0,0,0,2026,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {0,2,0,2,10,0,0,0,7,0,0,0,0,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {1,2,0,1,10,0,0,0,0,0,0,0,0,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {1,2,1,2,4,1,2,0,3,0,0,0,0,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {1,23,0,0,0,1,2,0,0,5,0,0,0,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {1,2,3,0,0,0,0,0,3,0,0,0,2026,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {1,90,91,0,1,0,0,0,7,0,0,0,0,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {1,90,91,8,1,0,0,0,7,0,0,0,0,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); rule = {1,26,0,8,0,1,0,0,0,0,0,0,0,0}; printrule(rule); forecasted = forecast(rule); print("%s\n\n",printintarray(forecasted,2)); }
#+RESULTS:
Compilation succeeded - 23 warning(s) "the" "2nd" "" "day" "" "of" "" "" "march" "" "of" "" "2026" "" getnextdate (single date) returns 2026-3-2 no more dates {2026,3,2} "the" "2nd" "" "tuesday" "" "of" "" "" "this month" "" "of" "" "2026" "" getnextdate (single date) returns 2026-7-14 {2026,7,14} "the" "2nd" "" "tuesday" "from the 10th" "of" "" "" "july" "" "of" "" "this year" "" getnextdate (single date) returns 2025-7-22 no more dates {2025,7,22} "every" "2nd" "" "monday" "from the 10th" "of" "" "" "this month" "" "of" "" "this year" "" [0] getnextdate (daycounter) returns 2025-7-21 [1] getnextdate (daycounter) returns 2025-8-4 [2] getnextdate (daycounter) returns 2025-8-18 [3] getnextdate (daycounter) returns 2025-9-1 [4] getnextdate (daycounter) returns 2025-9-15 {2025,7,21,2025,8,4,2025,8,18,2025,9,1,2025,9,15} "every" "2nd" "then single" "tuesday" "from the 4th" "of every" "2nd" "" "march" "" "of" "" "this year" "" [0] getnextdate (init) previous day 0, from day 4, weekday 2, nthday 2, month 7, year 2025 [0] getnextdate (countpermonth) dayoffset 1 + nth 14 is 15 [0] getnextdate (per month) returns 2025-7-15 [1] getnextdate (init) previous day 15, from day 4, weekday 2, nthday 2, month 7, year 2025 [1] getnextdate (countpermonth) dayoffset 15 + nth 7 is 22 [1] getnextdate (per month) returns 2025-7-22 [2] getnextdate (init) previous day 22, from day 4, weekday 2, nthday 2, month 7, year 2025 [2] getnextdate (countpermonth) dayoffset 22 + nth 14 is 36 [2] getnextdate (per month rollover) incremented +2 month is 2025-9-1 [2] getnextdate (per month rollover) new monthly offset is 2 [2] getnextdate (per month rollover) returns 2025-9-16 [3] getnextdate (init) previous day 16, from day 4, weekday 2, nthday 2, month 9, year 2025 [3] getnextdate (countpermonth) dayoffset 16 + nth 7 is 23 [3] getnextdate (per month) returns 2025-9-23 [4] getnextdate (init) previous day 23, from day 4, weekday 2, nthday 2, month 9, year 2025 [4] getnextdate (countpermonth) dayoffset 23 + nth 14 is 37 [4] getnextdate (per month rollover) incremented +2 month is 2025-11-1 [4] getnextdate (per month rollover) new monthly offset is 4 [4] getnextdate (per month rollover) returns 2025-11-18 {2025,7,15,2025,7,22,2025,9,16,2025,9,23,2025,11,18} "every" "23rd" "" "day" "" "of every" "2nd" "" "month" "from may" "of" "" "this year" "" [0] getnextdate (init) previous day 0, from day 0, weekday 0, nthday 23, month 7, year 2025 [0] getnextdate (countpermonth) dayoffset 0 + nth 23 is 23 [0] getnextdate (per month) returns 2025-7-23 [1] getnextdate (init) previous day 23, from day 0, weekday 0, nthday 23, month 7, year 2025 [1] getnextdate (countpermonth) dayoffset 23 + nth 23 is 46 [1] getnextdate (per month rollover) incremented +2 month is 2025-9-1 [1] getnextdate (per month rollover) returns 2025-9-23 [2] getnextdate (init) previous day 23, from day 0, weekday 0, nthday 23, month 9, year 2025 [2] getnextdate (countpermonth) dayoffset 23 + nth 23 is 46 [2] getnextdate (per month rollover) incremented +2 month is 2025-11-1 [2] getnextdate (per month rollover) returns 2025-11-23 [3] getnextdate (init) previous day 23, from day 0, weekday 0, nthday 23, month 11, year 2025 [3] getnextdate (countpermonth) dayoffset 23 + nth 23 is 46 [3] getnextdate (per month rollover) incremented +2 month is 2025-1-1 [3] getnextdate (per month rollover) returns 2025-1-23 [4] getnextdate (init) previous day 23, from day 0, weekday 0, nthday 23, month 1, year 2026 [4] getnextdate (countpermonth) dayoffset 23 + nth 23 is 46 [4] getnextdate (per month rollover) incremented +2 month is 2026-3-1 [4] getnextdate (per month rollover) returns 2026-3-23 {2025,7,23,2025,9,23,2025,11,23,2026,1,23,2026,3,23} "every" "2nd" "then 3rd" "day" "" "of" "" "" "march" "" "of" "" "2026" "" [0] getnextdate (daycounter) dayoffset 0, + nth 2 days is 2 (2025-7-1 + 2) [0] getnextdate (daycounter) returns 2026-3-3 [1] getnextdate (daycounter) returns 2026-3-6 [2] getnextdate (daycounter) returns 2026-3-8 [3] getnextdate (daycounter) returns 2026-3-11 [4] getnextdate (daycounter) returns 2026-3-13 {2026,3,3,2026,3,6,2026,3,8,2026,3,11,2026,3,13} "every" "90th" "then 91st" "day" "from the 1st" "of" "" "" "july" "" "of" "" "this year" "" [0] getnextdate (daycounter) dayoffset 0, + nth 90 days is 90 (2025-7-1 + 90) [0] getnextdate (daycounter) returns 2025-9-29 [1] getnextdate (daycounter) returns 2025-12-29 [2] getnextdate (daycounter) returns 2026-3-29 [3] getnextdate (daycounter) returns 2026-6-28 [4] getnextdate (daycounter) returns 2026-9-26 {2025,9,29,2025,12,29,2026,3,29,2026,6,28,2026,9,26} "every" "90th" "then 91st" "weekday on or before" "from the 1st" "of" "" "" "july" "" "of" "" "this year" "" [0] getnextdate (daycounter) dayoffset 0, + nth 90 days is 90 (2025-7-1 + 90) [0] getnextdate (daycounter) 2025-9-29 weekday is 1 [0] getnextdate (daycounter) returns 2025-9-29 [1] getnextdate (daycounter) 2025-12-29 weekday is 1 [1] getnextdate (daycounter) returns 2025-12-29 [2] getnextdate (daycounter) 2026-3-29 weekday is 7 [2] getnextdate (daycounter) returns 2026-3-27 [3] getnextdate (daycounter) 2026-6-26 weekday is 5 [3] getnextdate (daycounter) returns 2026-6-26 [4] getnextdate (daycounter) 2026-9-24 weekday is 4 [4] getnextdate (daycounter) returns 2026-9-24 {2025,9,29,2025,12,29,2026,3,27,2026,6,26,2026,9,24} "every" "26th" "" "weekday on or before" "" "of every" "" "" "month" "" "of" "" "this year" "" [0] getnextdate (init) previous day 0, from day 0, weekday 8, nthday 26, month 7, year 2025 [0] getnextdate (init) day 26 weekday is 6 [0] getnextdate (countpermonth) dayoffset 0 + nth 25 is 25 [0] getnextdate (per month) returns 2025-7-25 [1] getnextdate (init) previous day 25, from day 0, weekday 8, nthday 26, month 7, year 2025 [1] getnextdate (init) day 51 weekday is 0 [1] getnextdate (countpermonth) dayoffset 25 + nth 26 is 51 [1] getnextdate (per month rollover) incremented +1 month is 2025-8-1 [1] getnextdate (per month rollover) day 26 weekday is 2 [1] getnextdate (per month rollover) returns 2025-8-26 [2] getnextdate (init) previous day 26, from day 0, weekday 8, nthday 26, month 8, year 2025 [2] getnextdate (init) day 52 weekday is 0 [2] getnextdate (countpermonth) dayoffset 26 + nth 26 is 52 [2] getnextdate (per month rollover) incremented +1 month is 2025-9-1 [2] getnextdate (per month rollover) day 26 weekday is 5 [2] getnextdate (per month rollover) returns 2025-9-26 [3] getnextdate (init) previous day 26, from day 0, weekday 8, nthday 26, month 9, year 2025 [3] getnextdate (init) day 52 weekday is 0 [3] getnextdate (countpermonth) dayoffset 26 + nth 26 is 52 [3] getnextdate (per month rollover) incremented +1 month is 2025-10-1 [3] getnextdate (per month rollover) day 26 weekday is 7 [3] getnextdate (per month rollover) day 26 weekday 7 offset type is 1, offset is -2, nth is now 24 [3] getnextdate (per month rollover) returns 2025-10-24 [4] getnextdate (init) previous day 24, from day 0, weekday 8, nthday 26, month 10, year 2025 [4] getnextdate (init) day 50 weekday is 0 [4] getnextdate (countpermonth) dayoffset 24 + nth 26 is 50 [4] getnextdate (per month rollover) incremented +1 month is 2025-11-1 [4] getnextdate (per month rollover) day 26 weekday is 3 [4] getnextdate (per month rollover) returns 2025-11-26 {2025,7,25,2025,8,26,2025,9,26,2025,10,24,2025,11,26}
GETCHAR:vala:

Getting indices of unichar characters:

for (int x = 0; x < s.char_count(); x++) { ch = s.get_char(s.index_of_nth_char(x)); }

// test different methods of getting at characters... // by c.p.brown 2024 void main() { GLib.Intl.setlocale(ALL,""); unichar[] removeme = {'零','一','二','三','四','五','六','七','八','九'}; unichar ch; unichar qo = '«'; unichar qc = '»'; // iterate i @ for(), use cached index string s = "零1234五67«八»9零"; int i = 0; bool quoted = false; int64 tts = GLib.get_real_time(); int oldi = 0; for (int x = 0; s.get_next_char(ref i, out ch); x++) { if ((ch == qo) && (!quoted)) { quoted = true; } if ((ch == qc) && (quoted)) { quoted = false; } if ((!quoted) && (ch in removeme)) { s = s.splice(oldi,i,""); i = oldi; } oldi = i; } int64 tte = GLib.get_real_time(); print(s + "\n"); print("took %.2f microseconds\n\n",((double) (tte - tts))); // iterate i last s = "零1234五67«八»9零"; quoted = false; i = 0; int64 ttbs = GLib.get_real_time(); for (int x = 0; x < s.char_count(); x++) { if ((s.get_char(i) == qo) && (!quoted)) { quoted = true; } if ((s.get_char(i) == qc) && (quoted)) { quoted = false; } if ((!quoted) && (s.get_char(i) in removeme)) { s = s.splice(i,(i + s.get_char(i).to_utf8(null)),""); } s.get_next_char(ref i, out ch); } int64 ttbe = GLib.get_real_time(); print(s + "\n"); print("took %.2f microseconds\n\n",((double) (ttbe - ttbs))); // use index_of_nth_char instead of get_next_char s = "零1234五67«八»9零"; quoted = false; i = 0; int64 ttcs = GLib.get_real_time(); for (int x = 0; x < s.char_count(); x++) { i = s.index_of_nth_char(x); if ((s.get_char(i) == qo) && (!quoted)) { quoted = true; } if ((s.get_char(i) == qc) && (quoted)) { quoted = false; } if ((!quoted) && (s.get_char(i) in removeme)) { s = s.splice(i,(i + s.get_char(i).to_utf8(null)),""); } } int64 ttce = GLib.get_real_time(); print(s + "\n"); print("took %.2f microseconds\n\n",((double) (ttce - ttcs))); // use index_of_nth_char with previous index s = "零1234五67«八»9零"; quoted = false; i = 0; int64 ttds = GLib.get_real_time(); int o = 0; for (int x = 0; x < s.char_count(); x++) { i = s.index_of_nth_char(x+1); if ((s.get_char(o) == qo) && (!quoted)) { quoted = true; } if ((s.get_char(o) == qc) && (quoted)) { quoted = false; } if ((!quoted) && (s.get_char(o) in removeme)) { s = s.splice(o,i,""); } o = i; } int64 ttde = GLib.get_real_time(); print(s + "\n"); print("took %.2f microseconds\n\n",((double) (ttde - ttds))); // use index_of_nth_char with previous index and less get_char()s s = "零1234五67«八»9零"; quoted = false; i = 0; int64 ttes = GLib.get_real_time(); int p = 0; for (int x = 0; x < s.char_count(); x++) { unichar uh = s.get_char(p); i = s.index_of_nth_char(x+1); if ((uh == qo) && (!quoted)) { quoted = true; } else { if ((uh == qc) && (quoted)) { quoted = false; } } if ((!quoted) && (uh in removeme)) { s = s.splice(p,i,""); } p = i; } int64 ttee = GLib.get_real_time(); print(s + "\n"); print("took %.2f microseconds\n\n",((double) (ttee - ttes))); string pre = "unicode«善悪»_"; string suffixed = "_" + pre.substring(0,pre.index_of_nth_char(pre.char_count() - 1)); print("convert unicode prefix \"%s\" to suffix \"%s\"\n\n",pre, suffixed); string whs = " "; unichar whc = whs.get_char(0); print("int code for space is %d\n",((int) whc)); print("32 to char is \'%s\'\n",((char) 32).to_string()); print("32 to unichar is \'%s\'\n\n",((unichar) 32).to_string()); whs = "\t"; whc = whs.get_char(0); print("int code for tab is %d\n",((int) whc)); print("9 to char is \'%s\'\n",((char) 9).to_string()); print("9 to unichar is \'%s\'\n\n",((unichar) 9).to_string()); }
#+RESULTS:
123467«八»9 took 3.00 microseconds 123467«八»9 took 3.00 microseconds 123467«八»9 took 3.00 microseconds 123467«八»9 took 2.00 microseconds 123467«八»9 took 1.00 microseconds convert unicode prefix "unicode«善悪»_" to suffix "_unicode«善悪»" int code for space is 32 32 to char is ' ' 32 to unichar is ' ' int code for tab is 9 9 to char is ' ' 9 to unichar is ' '
tests:

// accessing unichar // by c.p.brown 2014 void main() { GLib.Intl.setlocale(ALL,""); string s = "\t«善悪»\n"; print("%s.length is %d\n",s,s.length); print("%s.char_count() is %d\n",s,s.char_count()); print("\n((string) s[i]):\n"); //for (int i = 0; i < s.char_count(); i++) { // print("s[%d] is \"%s\"\n",i,((string) s[i])); //} print("\t...segfault\n"); print("\ns[i].to_string():\n(invalid unicode chars)\n"); //for (int i = 0; i < s.char_count(); i++) { // print("s[%d] is \"%s\"\n",i,s[i].to_string()); //} print("\nget_char:\n(invalid unicode chars)\n"); //for (int i = 0; i < s.char_count(); i++) { // unichar c = s.get_char(i); // print("s.get_char(%d) is \'%u\' %d \"%s\"\n",i,c,((int) c),c.to_string()); //} print("\nget_char(index_of_nth_char()):\n"); for (int i = 0; i < s.char_count(); i++) { int n = s.index_of_nth_char(i); unichar c = s.get_char(n); print("s.get_char(index_of_nth_char(%d)) is \'%u\' %d \"%s\"\n",i,c,((int) c),c.to_string()); } print("\nget_next_char:\n"); unichar ch; int i = 0; int[] idxes = {0}; for (int x = 1; s.get_next_char(ref i, out ch); x++) { print("%d s.get_next_char(ref %d, out [unichar]) is \'%u\' %d \"%s\"\n",x,idxes[x-1],ch,((int) ch),ch.to_string()); idxes += i; } print("\nget_char(get_next_char()) @previous i:\n"); unichar cc; i = 0; idxes = {0}; for (int x = 1; s.get_next_char(ref i, out cc); x++) { unichar c = s.get_char(idxes[x-1]); print("%d s.get_char(%d) \'%u\' %d \"%s\"\n",x,idxes[x-1],c,((int) c),c.to_string()); idxes += i; } }
#+RESULTS:
«善悪» .length is 12 «善悪» .char_count() is 6 ((string) s[i]): ...segfault s[i].to_string(): (invalid unicode chars) get_char: (invalid unicode chars) get_char(index_of_nth_char()): s.get_char(index_of_nth_char(0)) is '9' 9 " " s.get_char(index_of_nth_char(1)) is '171' 171 "«" s.get_char(index_of_nth_char(2)) is '21892' 21892 "善" s.get_char(index_of_nth_char(3)) is '24746' 24746 "悪" s.get_char(index_of_nth_char(4)) is '187' 187 "»" s.get_char(index_of_nth_char(5)) is '10' 10 " " get_next_char: 1 s.get_next_char(ref 0, out [unichar]) is '9' 9 " " 2 s.get_next_char(ref 1, out [unichar]) is '171' 171 "«" 3 s.get_next_char(ref 3, out [unichar]) is '21892' 21892 "善" 4 s.get_next_char(ref 6, out [unichar]) is '24746' 24746 "悪" 5 s.get_next_char(ref 9, out [unichar]) is '187' 187 "»" 6 s.get_next_char(ref 11, out [unichar]) is '10' 10 " " get_char(get_next_char()) @previous i: 1 s.get_char(0) '9' 9 " " 2 s.get_char(1) '171' 171 "«" 3 s.get_char(3) '21892' 21892 "善" 4 s.get_char(6) '24746' 24746 "悪" 5 s.get_char(9) '187' 187 "»" 6 s.get_char(11) '10' 10 " "
GETCHARPACKET:vala:
// break string s into array of c items of up to n chars // by c.p.brown 2024 // set c to 0 to fit c to (s.char_count() / n), plus leftovers // dont set n to 0 string printstringarray (string[] s, int f) { // 0 1 2 3 4 5 6 7 8 string[] ii = {"", "\"", "{", "[", "[", "| ", "", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n", ",\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%s".printf(s[i]) + dd[f]); } o = (o + "%s".printf(s[s.length - 1]) + oo[f]); return o; } string[] charpacket (string s, int c, int n) { // c = number of packets // n = max chars per packet // c and n are exclusive, set c to 0 to use n int sc = s.char_count(); string[] pak = new string[c]; if (c == 0) { double sdv = ((double) sc) / ((double) n); int idv = (sc / n); if ((sdv - idv) > 0.0) { pak = new string[(idv + 1)]; } else { pak = new string[idv]; } } for (int i = 0; i < pak.length; i++) { pak[i] = ""; } int p = 0; int v = 0; sc = sc + 1; for (int x = 1; x < sc; x++) { unichar uh = s.get_char(p); pak[v] += uh.to_string(); p = s.index_of_nth_char(x); if ((x % n) == 0) { v += 1; if (v >= pak.length) { break; } } } return pak; } void main() { GLib.Intl.setlocale(ALL,""); string s = "零123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"; string[] p = charpacket(s,10,5); print("10 items, 5 chars:\n%s\n",printstringarray(p,2)); p = charpacket(s,4,3); print("4 items, 3 chars:\n%s\n",printstringarray(p,2)); p = charpacket(s,0,6); print("fit items, 6 chars:\n%s\n",printstringarray(p,2)); }
#+RESULTS:
10 items, 5 chars: {零1234,56789,ABCDE,FGHIJ,KLMNO,PQRST,UVWXY,Z,,} 4 items, 3 chars: {零12,345,678,9AB} fit items, 6 chars: {零12345,6789AB,CDEFGH,IJKLMN,OPQRST,UVWXYZ}
GETDUPES:vala:

Get rows of duplicate data in a column, using an array of row indices.
Used for re-sorting rows of data by column where the initial sort returns contiguous sequences of same-values, eg:

Unsorted data:
| I | DATA[] |
|---+--------|
| 0 | "CC"   |
| 1 | "AA"   |
| 2 | "EE"   |
| 3 | "DD"   |
| 4 | "DD"   |

Sorted as indices (X) of DATA:
| I | DATA[] | X[] | DATA[X[I]] |
|---+--------+-----+------------|
| 0 | "CC"   |   1 | (AA)       |
| 1 | "AA"   |   0 | (CC)       |
| 2 | "EE"   |   3 | (DD)       |
| 3 | "DD"   |   4 | (DD)       |
| 4 | "DD"   |   2 | (EE)       |

Contigious ranges of duplicates in DATA[X[I]]:
| I | DUPES[] | DATA[(((DUPES[I]>>32)*COLCOUNT)+COL)] |
|---+---------+---------------------------------------|
| 0 |    2..3 | "DD"                                  |

// get rows of duplicates in a column // uses an int[] of row indices instead of data[i] // indices[] may be in a different order to data[] // indices[] permits cheaper pre-partitioning for multithreading // // optionally search within a range of rows // // by c.p.brown, 2025 string printintarray (int[] s, int f) { string[] ii = {"", "\"", "{", "[", "[", "| ", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%d".printf(s[i]) + dd[f]); } o = (o + "%d".printf(s[s.length - 1]) + oo[f]); return o; } string printunrolledarray (int ind, string[] dat, int dsz, bool h, bool rnm, bool cnm, int m) { string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string spc = " "; bool domask = (m > -1); string dlm = " | "; int rcols = dsz; int rowcount = dat.length / dsz; if (rnm) { rcols = dsz+1; } int chc = 0; if (cnm) { chc = "%d".printf((dat.length - 1)).char_count(); } int[] cw = new int[rcols]; for (int i = 0; i < cw.length; i++) { cw[i] = 0; } if (rnm) { cw[0] = "%d".printf((dat.length/dsz)).char_count(); } if (h) { for (int i = 1; i < cw.length; i++) { string hh = "col %u".printf(i); cw[i] = hh.char_count(); } } string o = ""; string ln = ""; // get counts 1st for (int i = 0; i < dat.length; i++) { int c = (i % dsz); if (rnm) { c = c + 1; } int cc = 0; if (dat[i] != null) { cc = dat[i].char_count() + chc + 3; if (cc > 0) { int nwlat = dat[i].index_of("\n"); if (nwlat != -1) { cc = (nwlat + 3 + chc + 3); } } } cw[c] = int.max(cw[c],cc); } // print header if (h) { for (int i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } if (i == 0) { string hh = "#"; o = "%s%s%-*s".printf(o,dlm,cw[i],hh); } else { string hh = "col %d".printf(i-1); o = "%s%s%-*s".printf(o,dlm,cw[i],hh); } if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } // print line for (int i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s|-".printf(tabs); } ln = "%s%s%-*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-|\n".printf(ln); } } o = "%s%s".printf(o,ln); } int n = 0; int rcc = dsz + 1; for (int r = 0; r < rowcount; r++) { for (int c = 0; c < rcc; c++) { string datpart = ""; if (c == 0) { dlm = "%s| ".printf(tabs); o = "%s%s%s%.*s".printf(o,dlm,"%d".printf(r),(cw[0]-1),spc); } else { if (c == 1) { dlm = "%s| ".printf(tabs); } int j = (r * dsz) + (c - 1); int cc = dat[j].char_count(); if (dat[j] != null) { if (cc > 0) { datpart = dat[j]; int nwlat = dat[j].index_of("\n"); if (nwlat != -1) { //trunc = true; datpart = (dat[j].substring(0,nwlat) + "..."); cc = (nwlat + 3); } } else { cc = int.max(cc,1); } } dlm = " | "; string clen = " (%*d)".printf(chc,j); int padlen = (cw[c] - cc) - clen.char_count(); padlen = int.max(padlen,0); datpart = "%s%s%.*s".printf(datpart,clen,padlen,spc); if (domask) { bool ismasked = (((j % dsz) + m) == 0); if (ismasked) { n += 2; datpart = "\x1b[48;5;5m%s\x1b[0m".printf(datpart); } } o = "%s%s%s".printf(o,dlm,datpart); if (c == dsz) { dlm = " |\n"; o = "%s%s".printf(o,dlm); } } } } cw = null; return o; } string printallmyuint64bits (uint64 n) { string o = ""; for (int i = 63; i >= 0; i--) { o = o + "%lld".printf((n >> i) & 1); } return o; } string printallmyuint32bits (uint n) { string o = ""; for (int i = 31; i >= 0; i--) { o = o + "%ld".printf((n >> i) & 1); } return o; } uint64[] getduperowsbycol(int dsz, int[] dxs, int col, string[] s, int fr, int lr) { // note: start/end range packing sets an upper limit of about 4.2 billion rows // dxs is an array of row numbers, they are scrambled // dsz is the column count // fr is first row in dxs, lr is last row in dxs // to get an absolute index from dxs[i]: (dxs[i] * dsz) + col uint64[] o = {}; uint a = 0; // first row uint b = 0; // last row int r = -1; // recording? for (int i = (fr+1); i <= lr; i++) { int p = (dxs[i] * dsz) + col; int op = (dxs[i-1] * dsz) + col; //print("comparing col %d row %d s[%d] (%s) with col %d row %d s[%d] (%s)\n",col,dxs[i],p,s[p],col,dxs[i-1],op,s[op]); if (strcmp(s[p],s[op]) == 0) { if (r == -1) { // start recording print("\t1st duplicate row is %d...\n",(i-1)); a = (i-1); o += a; b = 0; r = 1; } else { // keep recording print("\tlatest duplicate row is %d...\n",(i-1)); b = (i-1); uint64 t = a; o[o.length - 1] = (t << 32) | b; } } else { // stop recording if (r == 1) { print("\tlast duplicate row is %d...\n",(i-1)); b = (i-1); uint64 t = a; o[o.length - 1] = (t << 32) | b; } a = 0; b = 0; r = -1; } if (i == lr) { // eof if (r == 1) { if (b != lr) { b = lr; uint64 t = a; o[o.length - 1] = (t << 32) | b; } } a = 0; b = 0; r = -1; } } return o; } void main() { string[] s = { "AA", "00", "{", "BB", "01", "{", "CC", "02", "{", "CC", "03", "}", "DD", "03", "[", "EE", "04", "[", "EE", "04", "]", "EE", "05", ")" }; int c = 0; int cc = 3; int[] dxs = {}; int rowcount = (s.length / cc); for (int i = 0; i < rowcount; i++) { dxs += i; } int rando = Random.int_range(0,(rowcount - 1)); uint n = 11; int m = 1; for (int i = (rowcount - 1); i >= 0; i--) { int t = dxs[i]; dxs[i] = dxs[rando]; dxs[rando] = t; n += 1; Random.set_seed(n); rando = Random.int_range(0,(rowcount - m)); m += 1; } print("\n%d SCRAMBLED ROW INDICES (dxs[]):\n",dxs.length); for (int i = 0; i < dxs.length; i++) { print("%d | %d | %s %s %s\n",i,dxs[i],s[dxs[i]*cc],s[(dxs[i]*cc)+1],s[(dxs[i]*cc)+2]); } print("\nRAW DATA:\n%s\n",printunrolledarray(1,s,3,true,true,true,-1)); for (int i = 0; i < cc; i++) { print("finding dupes in col %d rows %d to %d...\n",i,0,7); uint64[] dupes = getduperowsbycol(cc,dxs,i,s,0,7); for (int q = 0; q < dupes.length; q++) { uint64 bb = (dupes[q] & 0xFFFFFFFF); uint64 aa = (dupes[q] >> 32); int dxa = ((int) aa); int dxb = ((int) bb); print("\tduplicates of s[dxs[%d] (%d) *cc+1] (%d) \"%s\" in col %d starts on dxs[%d] (%lld), ends on dxs[%d] (%lld)\n",dxa,dxs[dxa],((dxs[dxa]*cc)+i),s[((dxs[dxa]*cc)+i)],i,dxa,aa,dxb,bb); } } }
#+RESULTS:
Compilation succeeded - 4 warning(s) 8 SCRAMBLED ROW INDICES (dxs[]): 0 | 3 | CC 03 } 1 | 2 | CC 02 { 2 | 7 | EE 05 ) 3 | 6 | EE 04 ] 4 | 0 | AA 00 { 5 | 4 | DD 03 [ 6 | 5 | EE 04 [ 7 | 1 | BB 01 { RAW DATA: | # | col 0 | col 1 | col 2 | |---+---------+---------+--------| | 0 | AA ( 0) | 00 ( 1) | { ( 2) | | 1 | BB ( 3) | 01 ( 4) | { ( 5) | | 2 | CC ( 6) | 02 ( 7) | { ( 8) | | 3 | CC ( 9) | 03 (10) | } (11) | | 4 | DD (12) | 03 (13) | [ (14) | | 5 | EE (15) | 04 (16) | [ (17) | | 6 | EE (18) | 04 (19) | ] (20) | | 7 | EE (21) | 05 (22) | ) (23) | finding dupes in col 0 rows 0 to 7... 1st duplicate row is 0... last duplicate row is 1... 1st duplicate row is 2... last duplicate row is 3... duplicates of s[dxs[0] (3) *cc+1] (9) "CC" in col 0 starts on dxs[0] (0), ends on dxs[1] (1) duplicates of s[dxs[2] (7) *cc+1] (21) "EE" in col 0 starts on dxs[2] (2), ends on dxs[3] (3) finding dupes in col 1 rows 0 to 7... finding dupes in col 2 rows 0 to 7... 1st duplicate row is 5... last duplicate row is 6... duplicates of s[dxs[5] (4) *cc+1] (14) "[" in col 2 starts on dxs[5] (5), ends on dxs[6] (6)
GETFILEPARTS:vala:
string[] getfilefileparts (GLib.File f) { string fileext = ""; string filename = ""; string filepath = ""; string fp = f.get_path(); int ii = fp.last_index_of("."); if (ii != -1) { fileext = fp.substring(ii+1); filename = fp.substring(0,ii); int qq = filename.last_index_of("/"); if (qq != -1) { filepath = filename.substring(0,qq); filename = filename.substring(qq + 1); } } return {f.get_parent().get_path(),filename,fileext}; } string[] getfilestringparts (string f) { string fileext = ""; string filename = ""; string filepath = ""; int ii = f.last_index_of("."); if (ii != -1) { fileext = f.substring(ii+1); filename = f.substring(0,ii); int qq = filename.last_index_of("/"); if (qq != -1) { filepath = filename.substring(0,qq); filename = filename.substring(qq + 1); } } return {filepath,filename,fileext}; } void main() { string sf = "./test/a/file/goes.here"; print("src...: %s\n",sf); GLib.File myfile = GLib.File.new_for_path(sf); string[] fp = getfilefileparts(myfile); print("file..: { %s }\n",string.joinv(", ",fp)); string[] sfp = getfilestringparts(sf); print("string: { %s }\n",string.joinv(", ",sfp)); }
#+RESULTS:
src...: ./test/a/file/goes.here file..: { /home/cpb/Desktop/txt/source/test/a/file, goes, here } string: { ./test/a/file, goes, here }
GEOCODE:busted::vala:

converting a labyrinth of id refs to csvs with the actual adresses in them.

Currently busted, needs a redesign to use postcodes instead of states, and sorting/indexing to hurry things up.


 |  # | 1. DETAIL | field               | col | 2. find missing in                      | find using          | in | output        |
 |----+-----------+---------------------+-----+-----------------------------------------+---------------------+----+---------------|
 |  0 |           | ADDRESS_SITE_NAME   |  30 | [state]_ADDRESS_STIE_psv.psv            | ADDRESS_SITE_PID    |  4 | sitename      |
 |  1 |           | NUMBER_FIRST        |  17 |                                         |                     |    | streetnumber  |
 |  2 |           | NUMBER_FIRST_SUFFIX |  18 |                                         |                     |    | streetsuffix  |
 |  3 |           | NUMBER_LAST         |  20 |                                         |                     |    | streetnumber2 |
 |  4 |           | NUMBER_LAST_SUFFIX  |  21 |                                         |                     |    | streetsuffix2 |
 |  5 |           | FLAT_NUMBER         |  10 |                                         |                     |    | unitnumber    |
 |  6 |           | FLAT_NUMBER_SUFFIX  |  11 |                                         |                     |    | unitsuffix    |
 |  7 |           | LEVEL_NUMBER        |  14 |                                         |                     |    | unitlevel     |
 |  8 |           | STREET_NAME         |  22 | [state]_STREET_LOCALITY_psv.psv         | STREET_LOCALITY_PID |  4 | streetname    |
 |  9 |           | STREET_TYPE_CODE    |  22 | [state]_STREET_LOCALITY_psv.psv         | STREET_LOCALITY_PID |  5 | streettype    |
 | 10 |           | STREET_SUFFIX_CODE  |  22 | [state]_STREET_LOCALITY_psv.psv         | STREET_LOCALITY_PID |  6 | streetsuffix  |
 | 11 |           | LOCALITY_NAME       |  24 | [state]_LOCALITY_psv.psv                | LOCALITY_PID        |  3 | city          |
 | 12 |           | (from file prefix)  |     |                                         |                     |    | state         |
 | 13 |           | POSTCODE            |  26 |                                         |                     |    | postcode      |
 | 14 |           | LATITUDE            |   0 | [state]_ADDRESS_DEFAULT_GEOCODE_psv.psv | ADDRESS_DETAIL_PID  |  6 | latitude      |
 | 15 |           | LONGITUDE           |   0 | [state]_ADDRESS_DEFAULT_GEOCODE_psv.psv | ADDRESS_DETAIL_PID  |  5 | longitude     |

// geocode test using Australian gov data // by cpbrown 2024 // // needs to be redesigned and better prepared for all string-in-string searching it has to do class geoc : Object { private int dsz; private string[] dat; private string[] hed; public int getcolbyname (string n) { if (strcmp(n,"ADDRESS_SITE_NAME") == 0) { return 0; } if (strcmp(n,"NUMBER_FIRST") == 0) { return 1; } if (strcmp(n,"NUMBER_FIRST_SUFFIX") == 0) { return 2; } if (strcmp(n,"NUMBER_LAST") == 0) { return 3; } if (strcmp(n,"NUMBER_LAST_SUFFIX") == 0) { return 4; } if (strcmp(n,"FLAT_NUMBER") == 0) { return 5; } if (strcmp(n,"FLAT_NUMBER_SUFFIX") == 0) { return 6; } if (strcmp(n,"LEVEL_NUMBER") == 0) { return 7; } if (strcmp(n,"STREET_NAME") == 0) { return 8; } if (strcmp(n,"STREET_TYPE_CODE") == 0) { return 9; } if (strcmp(n,"STREET_SUFFIX_CODE") == 0) { return 10; } if (strcmp(n,"LOCALITY_NAME") == 0) { return 11; } if (strcmp(n,"STATE") == 0) { return 12; } if (strcmp(n,"POSTCODE") == 0) { return 13; } if (strcmp(n,"LATITUDE") == 0) { return 14; } if (strcmp(n,"LONGITUDE") == 0) { return 15; } return -1; } public int datlen () { return dat.length; } public int datcolcount() { return dsz; } public int datrowcount() { return (dat.length / dsz); } public void datwrite (uint r, uint c, string d) { uint a = (r * dsz) + c; if (a < dat.length) { dat[a] = d; } } public void datwritebyname (int r, string n, string d) { int c = getcolbyname(n); if (c > -1) { int a = (r * dsz) + c; if (a < dat.length) { dat[a] = d; } } } public void datreplaceincolbyheadername (string h, string o, string n) { int c = getcolbyname(h); if (c > -1) { int rc = (dat.length / dsz); for (int a = c; a < rc; a += dsz) { if (strcmp(dat[a],o) == 0) { dat[a] = n; } } } } public void datappend (string d) { dat += d; } public void datappendrow (string[] d) { for (int i = 0; i < d.length; i++) { dat += d[i]; } } public int datfindreplaceincell (int r, int c, string o, string n) { int a = (r * dsz) + c; //print("\t\t\t comparing dat[%d] %s with %s...\n",a,dat[a],o); if (strcmp(dat[a],o) == 0) { //print("\t\t\t dat[%d] %s == %s, changing it to %s...\n",a,dat[a],o,n); dat[a] = n; return 1; } return 0; } public string datreadcellat (uint a) { return dat[a]; } public string datreadcell (int r, int c) { int a = (r * dsz) + c; return dat[a]; } public string datprintrow(int w) { int e = (dat.length / dsz); int r = w; string o = ""; if ((r >= 0) && (r < e)) { int a = (r * dsz); int b = (a + dsz); o = "row %d is {".printf(r); for (int i = a; i < (b-1); i++) { o = (o + dat[a] + ","); } o = (o + dat[b-1] + "}\n"); } return o; } public string datwritetofile (string p, string n) { string dir = p; string nom = n; string ext = "csv"; bool allgood = true; GLib.Dir dcr = null; try { dcr = Dir.open (dir, 0); } catch (Error e) { print("%s\n",e.message); File hdir = File.new_for_path(dir); try { hdir.make_directory_with_parents(); } catch (Error e) { print("%s\n",e.message); return ""; } } File hfile = File.new_for_path( dir.concat(nom,".",ext) ); FileOutputStream hose = hfile.replace( null, false, FileCreateFlags.PRIVATE ); for (int i = 0; i < hed.length; i++) { hose.write(hed[i].data); if (i < hed.length - 1) { hose.write(";".data); } else { hose.write("\n".data); } } for (int a = 0; a < dat.length; a++) { int c = (a % dsz); string cel = dat[a]; if (cel.contains("\n")) { cel = cel.replace("\n","\\n"); } if (cel.contains(";")) { if (cel.contains("\"")) { if (cel.contains("\'")) { cel = cel.replace("\'","<QUOTE>"); } cel = cel.replace("\"","\'"); } cel = ("\"" + cel + "\""); } string dlm = ";"; if (c == (dsz - 1)) { dlm = "\n"; if (a == (dat.length - 1)) { dlm = ""; } } try { hose.write((dat[a] + dlm).data); } catch (Error e) { print("%s\n",e.message); return ""; } } return hfile.get_path(); } public geoc () { dat = {}; dsz = 16; hed = { "sitename", "streetnumber", "streetsuffix", "streetnumber2", "streetsuffix2", "unitnumber", "unitsuffix", "unitlevel", "streetname", "streettype", "streetsuffix", "city", "state", "postcode", "latitude", "longitude" }; } } geoc statedata; int64[] detailsidx; // 00 sitename; // 01 streetnumber; // 02 streetsuffix; // 03 streetnumber2; // 04 streetsuffix2; // 05 unitnumber; // 06 unitsuffix // 07 unitlevel; // 08 streetname; // 09 streettype; // 10 streetsuffix; // 11 city; // 12 state; // 13 postcode; // 14 latitude; // 15 longitude; GLib.Mutex mutex; struct packet { public uint[] items; } string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } packet[] packseq ( uint[] x, uint c, bool q, int y, int z, int u) { //print("packseq started...\n"); // x = items to package // q = use items if true, use items count if false, eg {2,12,34,43} vs {0,1,2,3} // y = max packets, usually thread-count // z = nth item to package, eg {0,1,2,3,4,5} nth 2 is {0,2,4} // u = offset items, eg {0,1,2,3,4,5} offset 2 is {2,3,4,5}, {0,1,2,3,4,5} nth 2 offset 1 is {1,3,5} packet[] o = new packet[0]; uint[] srcitems = x; int maxpackets = y; int nthsrc = z; // nth int srcoffset = u; // offset uint srclen = c; uint[] sampleditems = {}; if (q) { srclen = srcitems.length; for (uint j = srcoffset; j < srclen; j += nthsrc) { sampleditems += srcitems[j]; } } else { for (uint j = srcoffset; j < srclen; j += nthsrc) { sampleditems += j; } } //print("sampleditems.length is %d, maxpackets is %d\n",sampleditems.length,maxpackets); if (sampleditems.length > maxpackets) { // y packets of n items uint itemsperpacket = (sampleditems.length / maxpackets); double neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); uint mxn = 0; //print("itemsperpacket is %u, neededpackets is %.1f\n",itemsperpacket,neededpackets); while (neededpackets > maxpackets) { if (mxn > 1000000) { //print("max packet count of 1 million packets reached, bailing...\n"); return o; } itemsperpacket += 1; neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); mxn += 1; } //print("itemsperpacket is %u, neededpackets is %.1f\n",itemsperpacket,neededpackets); uint np = (uint) neededpackets; uint dregs = sampleditems.length - (itemsperpacket * ((uint) neededpackets)); uint h = 0; if (dregs > 0) { h = 1; } if ((np + h) > 0) { o = new packet[(np+h)]; uint f = 0; for (uint i = 0; i < np; i++) { o[i] = new packet(); for (uint t = 0; t < itemsperpacket; t++) { o[i].items += sampleditems[f]; f += 1; } } if (dregs > 0) { o[np] = new packet(); for (uint t = 0; t < dregs; t++) { o[np].items += sampleditems[f]; f += 1; } } } } else { // one item per packet for (uint k = 0; k < sampleditems.length; k++) { packet oo = new packet(); oo.items += sampleditems[k]; o += oo; } } //print("packseq complete.\n"); return o; } public class gnafdetailsharvest : Object { string detailspath; uint thispacket; uint[] chunks; string state; public gnafdetailsharvest ( uint tp, uint[] chu, string fp, string st ) { thispacket = tp; chunks = chu; detailspath = fp; state = st; } public void eval(ref int[] threadcheck) { foreach (uint x in chunks) { GLib.FileStream dfstr = GLib.FileStream.open(detailspath,"r"); dfstr.seek(((long) detailsidx[x]),SET); string drl = dfstr.read_line(); if (drl == null) { break; } StringBuilder dline = new StringBuilder(""); dline.append(drl.strip()); if (dline.str.char_count() > 0) { string[] dcols = dline.str.split("|"); if (dcols.length >= 30) { mutex.lock(); statedata.datwrite(x,0,dcols[30]); // 00 sitename //if ((x % 10000) == 0) { print ("writing site id %s\n",dcols[30]); print("verify site id at row %d col %d %s\n",((int) x),0,statedata.datreadcell(((int) x),0)); } statedata.datwrite(x,1,dcols[17]); // 01 streetnumber statedata.datwrite(x,2,dcols[18]); // 02 streetsuffix statedata.datwrite(x,3,dcols[20]); // 03 streetnumber2 statedata.datwrite(x,4,dcols[21]); // 04 streetsuffix2 statedata.datwrite(x,5,dcols[10]); // 05 unitnumber statedata.datwrite(x,6,dcols[11]); // 06 unitsuffix statedata.datwrite(x,7,dcols[14]); // 07 unitlevel statedata.datwrite(x,8,dcols[22]); // 08 streetname statedata.datwrite(x,9,dcols[22]); // 09 streettype statedata.datwrite(x,10,dcols[22]); // 10 streetsuffix statedata.datwrite(x,11,dcols[24]); // 11 city statedata.datwrite(x,12,state); // 12 state statedata.datwrite(x,13,dcols[26]); // 13 postcode //if ((x % 10000) == 0) { print ("writing postcode %s\n",dcols[26]); print("verify postcode at row %d, col %d %s\n",((int) x),13,statedata.datreadcell(((int) x),13)); } statedata.datwrite(x,14,dcols[0]); // 14 latitude statedata.datwrite(x,15,dcols[0]); // 15 longitude mutex.unlock(); } } } threadcheck[thispacket] = 1; } } public class gnaffindandpreplace : Object { private int[,] checklist; private uint[] chunks; private uint thispacket; private string whatfile; public gnaffindandpreplace ( uint tp, uint[] chu, string wf, int[,] chk ) { thispacket = tp; chunks = chu; whatfile = wf; checklist = chk; } public void eval(ref int[] threadcheck) { foreach (uint x in chunks) { //if (x < 100) { print("processing dat row %u\n",x); } GLib.FileStream dfstr = GLib.FileStream.open(whatfile,"r"); string drl = dfstr.read_line(); if (drl == null) { continue; }; StringBuilder dheader = new StringBuilder(""); dheader.append(drl.strip()); string[] dheaders = dheader.str.split("|"); drl = dfstr.read_line(); int line = 1; while (drl != null) { StringBuilder dline = new StringBuilder(""); dline.append(drl.strip()); if (dline.str.char_count() > 0) { string[] dcols = dline.str.split("|"); if (dcols.length >= checklist[(checklist.length[0] - 1),2]) { // params: // {dat col, col containing search string, col containing replacement string} uint cell = (x * statedata.datcolcount()) + checklist[0,0]; if (strcmp(dcols[checklist[0,1]],statedata.datreadcellat(cell)) == 0) { int cc = 0; for (int c = 0; c < checklist.length[0]; c++) { mutex.lock(); cc += statedata.datfindreplaceincell( ((int) x),checklist[c,0],dcols[checklist[c,1]],dcols[checklist[c,2]] ); mutex.unlock(); } break; } } } line += 1; drl = dfstr.read_line(); } dfstr.rewind(); } threadcheck[thispacket] = 1; } } void main() { GLib.Intl.setlocale(ALL,""); mutex = GLib.Mutex(); int nthr = ((int) GLib.get_num_processors()); string[] nostates = { "ACT", "NSW", "NT", "QLD", "SA", "TAS", "VIC", "WA" }; string[] states = {"SA"}; string psvdir = "./psv/"; foreach (string state in states) { statedata = new geoc(); string detailspath = psvdir + state + "_ADDRESS_DETAIL_psv.psv"; GLib.File detailsfile = GLib.File.new_for_path(detailspath); string streetlocalitypath = psvdir + state + "_STREET_LOCALITY_psv.psv"; GLib.File streetlocalityfile = GLib.File.new_for_path(streetlocalitypath); string localitypath = psvdir + state + "_LOCALITY_psv.psv"; GLib.File localityfile = GLib.File.new_for_path(localitypath); string sitepath = psvdir + state + "_ADDRESS_SITE_psv.psv"; GLib.File sitefile = GLib.File.new_for_path(sitepath); string geocodepath = psvdir + state + "_ADDRESS_DEFAULT_GEOCODE_psv.psv"; GLib.File geocodefile = GLib.File.new_for_path(geocodepath); // 1. index details file, initialize data size int64 tts = GLib.get_monotonic_time(); detailsidx = {}; GLib.FileStream tfstr = GLib.FileStream.open(detailspath,"r"); StringBuilder tsbline = new StringBuilder(""); int64 accumulatedlines = 0; while (true) { string s = tfstr.read_line(); if (s == null) { break; } accumulatedlines += (s.length + 1); detailsidx += accumulatedlines; statedata.datappendrow( {"","","","","","","","","","","","","","","",""} ); } streetidx = {}; GLib.FileStream streetstream = GLib.FileStream.open(streetpath,"r"); accumulatedlines = 0; while (true) { string s = streetstream.read_line(); if (s == null) { break; } StringBuilder streetstring = new StringBuilder(""); streetstring.append(s); accumulatedlines += (s.length + 1); streetidx += accumulatedlines; streetids += ( {"","","","","","","","","","","","","","","",""} ); } int64 tte = GLib.get_monotonic_time(); print("indexing & memory allocation took %s\n",printusecs(tte-tts)); // 2. ransack details, populate 1st round of data tts = GLib.get_monotonic_time(); uint[] bq = new uint[0]; packet[] pak = packseq(bq,((uint) detailsidx.length),false,nthr,1,0); print("split indices into %d packets of %d items...\n",pak.length,pak[0].items.length); int[] threadcheck = new int[pak.length]; for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } string[] geocodes = new string[pak.length]; ThreadPool<gnafdetailsharvest> gs = new ThreadPool<gnafdetailsharvest>.with_owned_data( (gnafdetailsharvest) => {gnafdetailsharvest.eval(ref threadcheck); },nthr,false ); for (uint x = 0; x < pak.length; x++) { gs.add( new gnafdetailsharvest ( x, pak[x].items, detailspath, state ) ); } int lastindex = pak.length - 1; while (threadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } tte = GLib.get_monotonic_time(); print("base data collection took %s\n",printusecs(tte-tts)); string csvpath = ("./geocode/" + state + "/csv/"); string csvname = (state + "_temp.csv"); //statedata.datwritetofile(csvpath,csvname); // 3. get street locality data print("replacing street ids with street names...\n"); tts = GLib.get_monotonic_time(); bq = new uint[0]; pak = packseq(bq,((uint) statedata.datrowcount()),false,(nthr * 2),1,0); print("split indices into %d packets of %d items...\n",pak.length,pak[0].items.length); threadcheck = new int[pak.length]; for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } int[,] checklist = { {8,0,4},{9,0,5},{10,0,6} }; ThreadPool<gnaffindandpreplace> gfr = new ThreadPool<gnaffindandpreplace>.with_owned_data( (gnaffindandpreplace) => {gnaffindandpreplace.eval(ref threadcheck); },nthr,false ); for (uint x = 0; x < pak.length; x++) { gfr.add( new gnaffindandpreplace ( x, pak[x].items, streetlocalitypath, checklist ) ); } lastindex = pak.length - 1; while (threadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } tte = GLib.get_monotonic_time(); print("street data collection took %s\n",printusecs(tte-tts)); // 4. get site data print("replacing site ids with site names...\n"); tts = GLib.get_monotonic_time(); for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } checklist = { {0,0,4} }; for (uint x = 0; x < pak.length; x++) { gfr.add( new gnaffindandpreplace ( x, pak[x].items, sitepath, checklist ) ); } lastindex = pak.length - 1; while (threadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } tte = GLib.get_monotonic_time(); print("site data collection took %s\n",printusecs(tte-tts)); // 5. get city data print("replacing city ids with city names...\n"); tts = GLib.get_monotonic_time(); for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } checklist = { {11,0,3} }; for (uint x = 0; x < pak.length; x++) { gfr.add( new gnaffindandpreplace ( x, pak[x].items, localitypath, checklist ) ); } lastindex = pak.length - 1; while (threadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } tte = GLib.get_monotonic_time(); print("location data collection took %s\n",printusecs(tte-tts)); // 6. get geocode data print("replacing geocode ids with latitude and logngitude coords...\n"); tts = GLib.get_monotonic_time(); for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } checklist = { {14,3,6},{15,3,5} }; for (uint x = 0; x < pak.length; x++) { gfr.add( new gnaffindandpreplace ( x, pak[x].items, geocodepath, checklist ) ); } lastindex = pak.length - 1; while (threadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } tte = GLib.get_monotonic_time(); print("geolocation data collection took %s\n",printusecs(tte-tts)); // 7. write data csvpath = ("./geocode/" + state + "/csv/"); csvname = (state + ".csv"); print("writing to file %s%s\n",csvpath,csvname); statedata.datwritetofile(csvpath,csvname); statedata = null; print("data conversion for %s complete.\n\n",state); } }
#+RESULTS:
Compilation failed: 13 error(s), 4 warning(s)
GTKMENU:vala:

custom gtk menu

// custom menu example // menu items change state of other items // by c.p.brown 2025 bool doup; public class mymenu : Gtk.Box { public class mymenuitem : Gtk.Box { public int index; public Gtk.ToggleButton itemtoggle; public mymenuitem (int x, string n, bool t) { index = x; itemtoggle = new Gtk.ToggleButton(); itemtoggle.set_label(n); itemtoggle.set_active(t); itemtoggle.hexpand = true; this.append(itemtoggle); itemtoggle.toggled.connect((buh) => { if (doup) { doup = false; myparams mycontainercontainer = (myparams) this.get_ancestor(typeof(myparams)); mycontainercontainer.myselectionlabel.set_text(buh.get_label()); mymenu mycontainer = (mymenu) this.get_ancestor(typeof(mymenu)); mymenuitem mysibling = (mymenuitem) mycontainer.get_first_child(); while (mysibling != null) { if (mysibling.index != index) { mysibling.itemtoggle.active = false; } mysibling = (mymenuitem) mysibling.get_next_sibling(); } doup = true; } }); } } public mymenu (string[] d) { if ((d.length & 1) == 0) { int x = 0; for (int i = 0; i < d.length; i+=2) { bool tf = false; if (strcmp(d[i+1],"TRUE") == 0) { tf = true; } this.append(new mymenuitem (x, "%d_%s".printf(x,d[i]), tf)); x += 1; } } this.set_orientation(VERTICAL); this.set_spacing(2); } } public class myparams : Gtk.Box { public Gtk.Label myselectionlabel; public myparams () { this.set_orientation(VERTICAL); this.set_spacing(10); Gtk.MenuButton mymenubutton = new Gtk.MenuButton(); Gtk.Popover mymenupopover = new Gtk.Popover(); Gtk.ScrolledWindow mymenuscroll = new Gtk.ScrolledWindow(); myselectionlabel = new Gtk.Label(""); string[] mymenudata = { "ONE","FALSE", "TWO","FALSE", "THREE","FALSE", "FOUR","FALSE", "FIVE","FALSE", "SIX","FALSE", "SEVEN","FALSE", "EIGHT","FALSE", "NINE","FALSE", "TEN","FALSE" }; mymenu mymenuscrollbox = new mymenu(mymenudata); mymenupopover.set_child(mymenuscroll); mymenuscroll.set_child(mymenuscrollbox); mymenubutton.popover = mymenupopover; mymenubutton.hexpand = true; myselectionlabel.hexpand = true; Gtk.Requisition rq = new Gtk.Requisition(); Gtk.Requisition mq = new Gtk.Requisition(); mymenuscrollbox.get_preferred_size(out mq,out rq); mymenupopover.width_request = rq.width + 30; mymenupopover.height_request = int.min(360,rq.height); this.append(mymenubutton); this.append(myselectionlabel); } } int main(string[] args) { Gtk.Application myapp = new Gtk.Application("com.mytest.mytest",GLib.ApplicationFlags.DEFAULT_FLAGS); myapp.activate.connect (() => { doup = false; Gtk.ApplicationWindow mywin = new Gtk.ApplicationWindow(myapp); mywin.set_child(new myparams()); doup = true; mywin.default_width = 120; mywin.default_height = 120; mywin.present(); }); return myapp.run(args); }
HEAPSORT:vala:

Heapsort test.
This is derivative work, from various sources including Wikipedia & Rosetta Code.
Not entirely sure if this the correct way to do it, seems kinda onerous.

void hsnort (ref int[] p) { int64 tts = GLib.get_monotonic_time(); if (p.length > 0) { for (int k = 0; k < p.length; k++) { print("pre heapsort %d = %d\n",k,p[k]); } int e = p.length; int i = (e / 2) - 1; while (i >= 0) { int j = 0; int xx = 0; int yy = 0; while (i < e) { j = i; xx = (2 * i) + 1; yy = xx + 1; if ((xx < e) && (p[xx] > p[j])) { j = xx; } if ((yy < e) && (p[yy] > p[j])) { j = yy; } if (j == i) { break; } int t = p[i]; p[i] = p[j]; p[j] = t; // swap i = j; } i -= 1; } print("\n"); for (int k = 0; k < p.length; k++) { print("1st heapsort %d = %d\n",k,p[k]); } print("\n"); e = p.length - 1; while (e > 0) { int t = p[0]; p[0] = p[e]; p[e] = t; // swap i = 0; int j = 0; int xx = 0; int yy = 0; while (i < e) { j = i; xx = (2 * i) + 1; yy = xx + 1; if ((xx < e) && (p[xx] > p[j])) { j = xx; } if ((yy < e) && (p[yy] > p[j])) { j = yy; } if (j == i) { break; } t = p[i]; p[i] = p[j]; p[j] = t; // swap i = j; } e -= 1; } for (int k = 0; k < p.length; k++) { print("2nd heapsort %d = %d\n",k,p[k]); } } print("\n"); int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t heapsort took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t heapsort took %.2f microseconds\n",((double) (tte - tts))); } } void main() { int[] s = {5,4,1,9,8,6,0,3,7,2}; hsnort(ref s); }
#+RESULTS:
pre heapsort 0 = 5 pre heapsort 1 = 4 pre heapsort 2 = 1 pre heapsort 3 = 9 pre heapsort 4 = 8 pre heapsort 5 = 6 pre heapsort 6 = 0 pre heapsort 7 = 3 pre heapsort 8 = 7 pre heapsort 9 = 2 1st heapsort 0 = 9 1st heapsort 1 = 8 1st heapsort 2 = 6 1st heapsort 3 = 7 1st heapsort 4 = 5 1st heapsort 5 = 1 1st heapsort 6 = 0 1st heapsort 7 = 3 1st heapsort 8 = 4 1st heapsort 9 = 2 2nd heapsort 0 = 0 2nd heapsort 1 = 1 2nd heapsort 2 = 2 2nd heapsort 3 = 3 2nd heapsort 4 = 4 2nd heapsort 5 = 5 2nd heapsort 6 = 6 2nd heapsort 7 = 7 2nd heapsort 8 = 8 2nd heapsort 9 = 9 heapsort took 133.00 microseconds
GESTUREDRAG:gtk4::vala:
test gesture events...
// test gtk events in a bare-minimum application // by c.p.brown 2023 using Gtk; int main (string[] args) { Gtk.Application wut = new Gtk.Application ("com.test.test", GLib.ApplicationFlags.FLAGS_NONE); wut.activate.connect(() => { double[] mdn = {0.0,0.0}; double[] moo = {0.0,0.0}; double[] mof = {0.0,0.0}; Gtk.ApplicationWindow win = new Gtk.ApplicationWindow(wut); Gtk.DrawingArea im = new Gtk.DrawingArea(); im.margin_top = 10; im.margin_bottom = 10; im.margin_start = 10; im.margin_end = 10; Gtk.GestureDrag gd = new Gtk.GestureDrag(); im.add_controller(gd); im.set_draw_func((da,ctx,daw,dah) => { var bc = Gdk.RGBA(); bc.parse("#112633"); ctx.set_source_rgba(bc.red,bc.green,bc.blue,1); ctx.paint(); bc.parse("#DDDD88DD"); ctx.set_source_rgba(bc.red,bc.green,bc.blue,bc.alpha); ctx.set_line_width(2.0); ctx.move_to(mof[0],0.0); ctx.line_to(mof[0],dah); ctx.stroke(); ctx.move_to(0.0,mof[1]); ctx.line_to(daw,mof[1]); ctx.stroke(); string inf = "x = %0.2f, y = %0.2f".printf(moo[0],moo[1]); ctx.select_font_face("Monospace",Cairo.FontSlant.NORMAL,Cairo.FontWeight.BOLD); Cairo.TextExtents extents; ctx.text_extents (inf, out extents); ctx.set_source_rgba(((float) 0.9),((float) 0.9),((float) 0.4),((float) 0.9)); ctx.move_to(20.0,(dah - extents.height)); ctx.show_text(inf); }); gd.drag_begin.connect((event,x,y) => { print("device button is %u\n", event.get_current_button()); mdn = {x,y}; }); gd.drag_update.connect((event,x,y) => { moo = {x,y}; mof = {mdn[0] + x,mdn[1] + y}; im.queue_draw(); }); win.default_width = 300; win.default_height = 300; win.set_child(im); win.present(); }); return wut.run(args); }
#+RESULTS:

GOLDPACT:vala:

Check char pair balance.
Used to validate expressions before parsing them.

// char-pair check // unroll to visualize, assuming the string passes the check // by c.p.brown, 2024 // // TODO: check for escape chars, quotes and commented lines string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.3f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.3f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.3f m".printf((((double) u) / 60000000.0)); } return us; } bool goldpact(string s, unichar c) { StringBuilder xpr = new StringBuilder(""); xpr.append(s.strip()); unichar a = '('; unichar b = ')'; bool isidentical = false; if (c == '[') { a = '['; b = ']'; } else { if (c == '{') { a = '{'; b = '}'; } else { if (c == '<') { a = '<'; b = '>'; } else { if (c == '\"') { a = '\"'; b = '\"'; isidentical = true; } else { if (c == '\'') { a = '\''; b = '\''; isidentical = true; } else { print("invalid character %s\n",c.to_string()); return false; } } } } } int ii = xpr.str.index_of_char(a); int oo = xpr.str.index_of_char(b,(ii+1)); int i = 0; int o = 0; int pos = 0; int n = 0; if ((ii != -1) && (oo != -1)) { if (ii == 0) { i += 1; } while (pos != -1) { int nxt = xpr.str.index_of_char(a,(pos+1)); if (!isidentical) { oo = xpr.str.index_of_char(b,pos+1); if ((nxt != -1) && (nxt < oo)) { i += 1; } pos = nxt; if (oo != -1) { if (oo < nxt) { pos = oo; o += 1; } if (nxt == -1) { pos = oo; o += 1; } } } else { if (nxt != -1) { i += 1; } pos = nxt; } n += 1; } print("goldpact ran %d iterations\n",n); if (isidentical) { //i -= 1; return ((i & 1) == 0); } else { return (i == o); } } return false; } string unroll (string s, unichar c) { StringBuilder xpr = new StringBuilder(""); xpr.append(s.strip()); unichar a = '('; unichar b = ')'; bool isidentical = false; if (c == '[') { a = '['; b = ']'; } else { if (c == '{') { a = '{'; b = '}'; } else { if (c == '<') { a = '<'; b = '>'; } else { if (c == '\"') { a = '\"'; b = '\"'; isidentical = true; } else { if (c == '\'') { a = '\''; b = '\''; isidentical = true; } else { return "invalid character %s".printf(c.to_string()); } } } } } int ii = xpr.str.index_of_char(a); int oo = xpr.str.index_of_char(b,(ii+1)); int pos = 0; int i = 0; string u = ""; string tabs = ""; int n = 0; while (pos != -1) { int nxt = xpr.str.index_of_char(a,pos); if (!isidentical) { int npos = nxt; oo = xpr.str.index_of_char(b,pos); if ((nxt != -1) && (nxt < oo)) { i += 1; npos += 1; u = u + tabs + xpr.str.substring(pos,(npos-pos)).strip()+ "\n"; tabs = ("%*s").printf(i,"").replace(" ","\t"); //u = u + tabs + "%s".printf(a.to_string()) + "\n"; } if (oo != -1) { if ((oo < nxt) || (nxt == -1)) { i -= 1; string thisline = xpr.str.substring(pos,(oo-pos)).strip(); if (thisline.char_count() > 0) { u = u + tabs + xpr.str.substring(pos,(oo-pos)).strip() + "\n"; } tabs = ("%*s").printf(i,"").replace(" ","\t"); u = u + tabs + "%s".printf(b.to_string()) + "\n"; pos = oo + 1; } else { pos = npos; } } else { pos = npos; } } else { if (nxt != -1) { n += 1; if ((n & 1) != 0) { i += 1; u = u + tabs + xpr.str.substring(pos,((nxt+1)-pos)).strip()+ "\n"; tabs = ("%*s").printf(i,"").replace(" ","\t"); } else { i -= 1; string thisline = xpr.str.substring(pos,(nxt-pos)).strip(); if (thisline.char_count() > 0) { u = u + tabs + xpr.str.substring(pos,(nxt-pos)).strip() + "\n"; } tabs = ("%*s").printf(i,"").replace(" ","\t"); u = u + tabs + "%s".printf(b.to_string()) + "\n"; } pos = nxt + 1; } else { break; } } } return u; } void main () { GLib.Intl.setlocale(ALL,""); string s = "[ MUL X [PARAM 0.1 0.0 1.0 ][ DIV [ COMP A ] X ]]"; bool isbalanced = goldpact(s,'['); if (isbalanced) { print("expression has balanced brackets:\n%s\n",unroll(s,'[')); } else { print("expression is unbalanced: %s\n\n",s); } s = "[ MUL X [PARAM 0.1 0.0 1.0 [ DIV [ COMP A ] X ]]"; isbalanced = goldpact(s,'['); if (isbalanced) { print("expression has balanced brackets:\n%s\n",unroll(s,'[')); } else { print("expression is unbalanced: %s\n\n",s); } s = "[[[MUL X[PARAM 0.1 0.0 1.0][DIV[COMP A]X]]]]"; isbalanced = goldpact(s,'['); if (isbalanced) { print("expression has balanced brackets:\n%s\n",unroll(s,'[')); } else { print("expression is unbalanced: %s\n\n",s); } s = "[[[[[[MUL X[PARAM 0.1 0.0 1.0 DIV[][]]]COMP A]X][]]]"; isbalanced = goldpact(s,'['); if (isbalanced) { print("expression has balanced brackets:\n%s\n",unroll(s,'[')); } else { print("expression is unbalanced: %s\n\n",s); } s = "COMMENT[][][MUL X[PARAM 0.1 0.0 1.0][DIV[COMP A []][]X]][][]"; isbalanced = goldpact(s,'['); if (isbalanced) { print("expression has balanced brackets:\n%s\n",unroll(s,'[')); } else { print("expression is unbalanced: %s\n\n",s); } s = "a \"string \"in a\" string\" ?"; isbalanced = goldpact(s,'\"'); if (isbalanced) { print("text has balanced quotes, however unrolling it isn't useful:\n%s\n",unroll(s,'\"')); } else { print("string quotes are unbalanced: %s\n\n",s); } print("\n\ncheck a 4305-line file:\n"); StringBuilder sb = new StringBuilder(""); string testfile = "test_zui.vala"; string ff = Path.build_filename ("./", testfile); GLib.File og = File.new_for_path(ff); if (og.query_exists() == true) { try { uint8[] ca; string oe; og.load_contents (null, out ca, out oe); sb.append((string) ca); } catch (Error e) { print ("\tfailed to read %s: %s\n", og.get_path(), e.message); } } int64 tts = GLib.get_monotonic_time(); isbalanced = goldpact(sb.str,'{'); if (isbalanced) { print("test_zui.vala is balanced\n"); } else { print("test_zui.vala is not balanced, probably because we're not looking for commented, quoted or escaped curlies here\n"); } int64 tte = GLib.get_monotonic_time(); print("goldpact stress test took %s\n",printusecs(tte-tts)); }
#+RESULTS:
goldpact ran 8 iterations expression has balanced brackets: [ MUL X [ PARAM 0.1 0.0 1.0 ] [ DIV [ COMP A ] X ] ] goldpact ran 7 iterations expression is unbalanced: [ MUL X [PARAM 0.1 0.0 1.0 [ DIV [ COMP A ] X ]] goldpact ran 12 iterations expression has balanced brackets: [ [ [ MUL X[ PARAM 0.1 0.0 1.0 ] [ DIV[ COMP A ] X ] ] ] ] goldpact ran 19 iterations expression is unbalanced: [[[[[[MUL X[PARAM 0.1 0.0 1.0 DIV[][]]]COMP A]X][]]] goldpact ran 21 iterations expression has balanced brackets: COMMENT[ ] [ ] [ MUL X[ PARAM 0.1 0.0 1.0 ] [ DIV[ COMP A [ ] ] [ ] X ] ] [ ] [ ] goldpact ran 5 iterations text has balanced quotes, however unrolling it isn't useful: a " string " in a" string " check a 4305-line file: goldpact ran 2071 iterations text_zui.vala is not balanced, probably because we're not looking for commented or escaped curlies here goldpact stress test took 252 μs
HEX:vala:

print hex

void main (string[] args) { int huh = 0xFFAA88; print("#%X\n",huh); }
INHERIT:vala:

In Vala simply writing an Object inside another Object won't allow it to access stuff in the outer Object as they're two distinct Objects. Technically both Objects should be written separately and the 'parent' Object, or a subset of its properties, is passed as args to the 'child' Object, or in Gtk use parentobject myparent (parentobject) = this.get_ancestor(typeof(parentobject)); where the UI Objects are nested. An example of get_ancestor() can be seen in GTKMENU where a clicked menu-item changes other menu-items in its parent container, and also a label in a 'parameters' Object to which the menu-button belongs.

INTERLEAVE:vala:

Combine and sort two index arrays of a string array.
Used to combine segments of a multithreaded sort.

// sorted interleave of two sets of indices of a scrambled string array, // each sorted by what-goes-here: // 00 BB 01 AA // 01 AA = 00 BB // 02 CC 02 CC // // by c.p.brown 2025 GLib.Mutex mutex; string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } class opsort : Object { public string[] dat; public int dsz; public int[] dxs; public void dxsinit () { dxs.resize(0); for (int i = 0; i < dat.length; i++) { dxs += i; } } public void mergedxs (int[] aa, int[] bb, int ins) { // indices are what-goes-here // I J D S a // 00 0 FF AA 1 // 01 1 AA DD 2 // 02 2 DD FF 0 // // 03 0 CC BB 5 // 04 1 EE CC 3 // 05 2 BB EE 4 // incremental filing using markers a and b // | dat[aa[a]]<dat[bb[b]] | AA<BB | X[0]=1 D[1] is AA | a++ (1 DD) // | dat[aa[a]]<dat[bb[b]] | DD<BB | X[1]=5 D[5] is BB | b++ (1 CC) // | dat[aa[a]]<dat[bb[b]] | DD<CC | X[2]=3 D[3] is CC | b++ (2 EE) // | dat[aa[a]]<dat[bb[b]] | DD<EE | X[3]=2 D[2] is DD | a++ (2 FF) // | dat[aa[a]]<dat[bb[b]] | FF<EE | X[4]=4 D[4] is EE | b++ EOF // add remainder of a or b in sequence // | dat[aa[a]] | | X[5]=0 D[0] is FF | a++ EOF // I | 00 01 02 || 03 04 05 | array index // X | 01 02 00 || 05 03 04 | sorted index // D | FF AA DD || CC EE BB | data @ sorted index // | a || b | offset // | ^^ <= ^^ | compare // R | AA || | append // + | ++ a || b | increment // // I | 00 01 02 || 03 04 05 | // X | 01 02 00 || 05 03 04 | // D | FF AA DD || CC EE BB | // | a || b | // | ^^ <= ^^ | // R | AA BB || | // + | a || ++ b | // // I | 00 01 02 || 03 04 05 | // X | 01 02 00 || 05 03 04 | // D | FF AA DD || CC EE BB | // | a || b | // | ^^ <= ^^ | // R | AA BB CC || | // + | a || ++ b | // // I | 00 01 02 || 03 04 05 | // X | 01 02 00 || 05 03 04 | // D | FF AA DD || CC EE BB | // | a || b | // | ^^ <= ^^ | // R | AA BB CC || DD | // + | ++ a || b | // // I | 00 01 02 || 03 04 05 | // X | 01 02 00 || 05 03 04 | // D | FF AA DD || CC EE BB | // | a || b | // | ^^ <= ^^ | // R | AA BB CC || DD EE | // + | a || ++ | b is done, just collect a // // I | 00 01 02 || 03 04 05 | // X | 01 02 00 || 05 03 04 | // D | FF AA DD || CC EE BB | // | a || | // | ^^ || | just collect X at a // R | AA BB CC || DD EE FF | // + | ++ || | // 0 1 2 3 4 5 6 7 8 9 | 10 // ^ | insert at 2 // 0 1 2 3 0 1 2 3 | 8 (4+4) // | 2+8 <= 10 int a = 0; int b = 0; int x = ins; int al = aa.length; int bl = bb.length; if ((ins + (al + bl)) <= dxs.length) { while ((a < al) && (b < bl)) { if (strcmp(dat[aa[a]],dat[bb[b]]) <= 0) { dxs[x] = aa[a]; a += 1; } else { dxs[x] = bb[b]; b += 1; } x += 1; } while (a < al) { dxs[x] = aa[a]; a += 1; x += 1; } while (b < bl) { dxs[x] = bb[b]; b += 1; x += 1; } } } } void main() { GLib.Intl.setlocale(ALL,""); mutex = GLib.Mutex(); string[] s = { "AA", "HH", "FF", "II", "KK", "CC", "DD", "LL", "EE", "BB", "GG", "JJ" }; // 00 AA 00 AA // 01 HH 05 CC // 02 FF 02 FF // 03 II 01 HH // 04 KK 03 II // 05 CC 04 KK // // 06 DD 09 BB // 07 LL 06 DD // 08 EE 08 EE // 09 BB 10 GG // 10 GG 11 JJ // 11 JJ 07 LL int[] a = { 0,5,2,1,3,4 }; int[] b = { 9,6,8,10,11,7 }; opsort o = new opsort(); o.dat = s; o.dsz = 1; o.dxsinit(); print("\nSCRAMBLED DATA\n"); for (int i = 0; i < o.dat.length; i++) { print("o.dat[%02d] is %s\n",i,o.dat[i]); } print("\nSORTED INDEX PACKETS\n"); for (int i = 0; i < a.length; i++) { print("a[%d] is %02d (%s)\n",i,a[i],o.dat[a[i]]); } print("\n"); for (int i = 0; i < b.length; i++) { print("b[%d] is %02d (%s)\n",i,b[i],o.dat[b[i]]); } int64 tts = GLib.get_monotonic_time(); o.mergedxs (a, b, 0); int64 tte = GLib.get_monotonic_time(); print("\nmergedxs took %s\n",printusecs(tte-tts)); print("\nDXS\n"); for (int i = 0; i < o.dxs.length; i++) { print("o.dxs[%02d] is %02d dat[%02d] is %s\n",i,o.dxs[i],o.dxs[i],o.dat[o.dxs[i]]); } print("\n"); }
#+RESULTS:
SCRAMBLED DATA o.dat[00] is AA o.dat[01] is HH o.dat[02] is FF o.dat[03] is II o.dat[04] is KK o.dat[05] is CC o.dat[06] is DD o.dat[07] is LL o.dat[08] is EE o.dat[09] is BB o.dat[10] is GG o.dat[11] is JJ SORTED INDEX PACKETS a[0] is 00 (AA) a[1] is 05 (CC) a[2] is 02 (FF) a[3] is 01 (HH) a[4] is 03 (II) a[5] is 04 (KK) b[0] is 09 (BB) b[1] is 06 (DD) b[2] is 08 (EE) b[3] is 10 (GG) b[4] is 11 (JJ) b[5] is 07 (LL) mergedxs took 1 μs DXS o.dxs[00] is 00 dat[00] is AA o.dxs[01] is 09 dat[09] is BB o.dxs[02] is 05 dat[05] is CC o.dxs[03] is 06 dat[06] is DD o.dxs[04] is 08 dat[08] is EE o.dxs[05] is 02 dat[02] is FF o.dxs[06] is 10 dat[10] is GG o.dxs[07] is 01 dat[01] is HH o.dxs[08] is 03 dat[03] is II o.dxs[09] is 11 dat[11] is JJ o.dxs[10] is 04 dat[04] is KK o.dxs[11] is 07 dat[07] is LL
ISDIGIT:vala:

Text to numbers

void main() { if ('-'.isdigit()) { print("\"-\" is a digit\n"); } if ('+'.isdigit()) { print("\"+\" is a digit\n"); } if ('/'.isdigit()) { print("\"/\" is a digit\n"); } if (','.isdigit()) { print("\",\" is a digit\n"); } if ('\''.isdigit()) { print("\"\'\" is a digit\n"); } print("\"+1\" is %d\n",int.parse("+1")); print("\"-1\" is %d\n",int.parse("-1")); print("\"1/1/11\" is %d\n",int.parse("1/1/11")); print("\"$1\" is %d\n",int.parse("$1")); print("\"$1\" is %.2f\n",double.parse("$1")); print("\"1000\'000\" is %d\n",int.parse("1000\'000")); print("\"1.0\" is %.2f\n",double.parse("1.0")); print("\"1,0\" is %.2f\n",double.parse("1,0")); print("\"1:1\" is %d\n",int.parse("1:0")); print("\"10_\" is %d\n",int.parse("10_")); print("\"10A\" is %d\n",int.parse("10A")); print("\"1.0_ABC_2.0\" is %d\n",int.parse("1.0_ABC_2.0")); }
#+RESULTS:
"+1" is 1 "-1" is -1 "1/1/11" is 1 "$1" is 0 "$1" is 0.00 "1000'000" is 1000 "1.0" is 1.00 "1,0" is 1.00 "1:1" is 1 "10_" is 10 "10A" is 10 "1.0_ABC_2.0" is 1
IXLIC:fluff::vala:

Translator test

public class ixlic { // experiment in translating stuff // desired usage is: ixl.get("word to translate",langcode); // where langcode is english if 0, then whatever is in the list // change langcode at runtime, somehow trigger redraw of everything in the ui (test later) // by c.p.brown 2024 // fold array after 4 (5 cols) // | 00 | "allgood" | "translation1" | "translation2" | "translation3" | // | 07 | "hello" | "translation1" | "translation2" | "translation3" | // | 07 | "huh" | "translation1" | "translation2" | "translation3" | // | 22 | "what" | "translation1" | "translation2" | "translation3" | // 3 3 3 3 3 3 3 | k // 0 1 2 3 4 5 6 | i // 0 1 1 2 4 4 4 | dat[i] // 0 1 2 3 3 3 3 | i if k >= dat[i] // 2 2 2 2 2 2 2 | k // 0 1 2 3 4 5 6 | i // 0 1 1 3 4 4 5 | dat[i] // 0 1 2 2 2 2 2 | i if k >= dat[i] private string[] dat; private int csz; public int datseek (string k) { int preep = 0; for (int i = 0; i < dat.length; i += csz) { if (strcmp(k,dat[i]) > 0) {preep = (i+csz);} if (dat[i] == k) { return i; } } print("\n"); return preep; } public string get (string s, int l) { if (l == 0) { return s; } if ((l + 1) >= csz) { return s; } string u = s.up(); string k = "%d".printf(((int) u.get_char(0)) - 65); int ii = datseek(k); if (ii == -1) { return s; } ii = int.min(ii,(csz-1)); for (int w = ii; w < dat.length; w += csz) { if (strcmp(u,dat[w+1]) == 0) { return dat[w+(l+1)]; } } return s; } public bool put (string s, int l, string t) { if (l == 0) { return false; } if ((l + 1) >= csz) { return false; } string u = s.up(); string k = "%d".printf(((int) u.get_char(0)) - 65); int ii = datseek(k); if (ii == -1) { return false; } for (int w = ii; w < dat.length; w += csz) { if (strcmp(u,dat[w+1]) == 0) { dat[w+(l+1)] = t; return true; } } return false; } public void add (string s, string[] l) { string u = s.up(); string k = "%d".printf(((int) u.get_char(0)) - 65); string[] w = new string[csz]; w[0] = k; w[1] = u; for (int i = 2; i < csz; i++) { w[i] = l[i-2]; } int p = datseek(k); if (p == -1) { p = 0; } int a = p * csz; if ((p < 0) || (p > csz)) { int r = (int) (dat.length / csz); a = (r * csz); } int n = csz - 1; dat.resize(dat.length + csz); for (int i = (dat.length - 1); i >= a; i--) { if (i >= (a + csz)) { dat[i] = dat[i - csz]; } else { if (w[n] == null) { dat[i] = ""; n -= 1; } else { dat[i] = w[n]; n -= 1; } } } } public ixlic () { dat = {}; // index, english, 3 translations csz = 5; } } void main() { GLib.Intl.setlocale(ALL,""); string[] lang = { "ENG", "UGH", "REE", "NUG" }; ixlic ixl = new ixlic(); ixl.add("hello",{"snort","gurgle","MMMOOOO","grunt"}); print("Translate \"hello\" to %s... \"%s\"\n",lang[0],ixl.get("hello",0)); print("Translate \"hello\" to %s... \"%s\"\n",lang[1],ixl.get("hello",1)); ixl.add("what",{"squeal","plop","NGRRGGH","groan"}); print("Translate \"what\" to %s... \"%s\"\n",lang[3],ixl.get("what",3)); print("I mean \"what\"... "); if (ixl.put("what",3,"ẁĥąţ")) { print("\"%s\"\n\nFine whatever.\n\n",ixl.get("what",3)); } }
#+RESULTS:
Translate "hello" to ENG... "hello" Translate "hello" to UGH... "snort" Translate "what" to NUG... "NGRRGGH" I mean "what"... "ẁĥąţ" Fine whatever. Fucken weirdo.
IXLICNUMS_GFN:vala:

Convert numbers to its English description.

// numbers to english // by c.p.brown 2024 string ands (int n) { if (n > 0) { return " and "; } return ""; } string andhypers(int n, int l) { string[] hypers = {" "," hundred"," thousand"," million"," billion"," trillion"}; if (n > 0) { return hypers[l]; } return ""; } int isum (ref int[] n) { int r = 0; foreach (int i in n) { r += i; } return r; } string[] maketens () { string[] ones = { "", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine" }; string[] reftens = { "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen" }; string[] tentens = { "twenty", "thirty", "fourty", "fifty", "sixty", "seventy", "eighty", "ninety" }; string[] tens = {}; for (int t = 0; t < ones.length; t++) { tens += ones[t]; } for (int t = 0; t < reftens.length; t++) { tens += reftens[t]; } for (int tt = 0; tt < tentens.length; tt++) { for (int t = 0; t < reftens.length; t++) { if (t > 0) { tens += (tentens[tt] + "-" + ones[t]); } else { tens += tentens[tt]; } } } return tens; } string inglicnums (int64 n, bool ano, ref string[] tens) { string[] ones = { "", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine" }; int[] p = new int[16]; string sn = "%lld".printf(n); int snc = sn.char_count(); if (snc == 1) { if (n == 0) { if (ano) { return ("%lld".printf(n) + " (" + ones[n] + ")"); } else { return ones[n]; } } } int ofs = 16 - snc; for (int c = 0; c < snc; c++) { if (c < 16) { p[c+ofs] = sn.get_char(c).digit_value(); } else { if (ano) { return ("%lld".printf(n) + " (squillions)"); } else { return "squillions"; } } } if (isum(ref p) == 0) { if (ano) { return ("%lld".printf(n) + " (zero)"); } else { return "zero"; } } if (snc > 15) { if (ano) { return ("%lld".printf(n) + " (squillions)"); } else { return "squillions"; } } if (snc == 1) { if (n == 0) { if (ano) { return ("%lld".printf(n) + " (" + ones[n] + ")"); } else { return ones[n]; } } } string incq = ""; if (snc > 1) { incq = tens[((p[14]*10)+p[15])]; } if (snc > 2) { incq = (andhypers(p[13],0) + ones[p[13]] + andhypers(p[13],1) + ands(p[14]+p[15]) + incq); } if (snc > 3) { incq = (tens[((p[11]*10)+p[12])] + andhypers((p[11]+p[12]+p[10]),2) + incq); } if (snc > 5) { incq = (andhypers(p[10],0) + ones[p[10]] + andhypers(p[10],1) + ands(p[11]+p[12]) + incq); } if (snc > 6) { incq = (tens[((p[8]*10)+p[9])] + andhypers((p[8]+p[9]+p[7]),3) + incq); } if (snc > 8) { incq = (andhypers(p[7],0) + ones[p[7]] + andhypers(p[7],1) + ands(p[8]+p[9]) + incq); } if (snc > 9) { incq = (tens[((p[5]*10)+p[6])] + andhypers((p[5]+p[6]+p[4]),4) + incq); } if (snc > 11) { incq = (andhypers(p[4],0) + ones[p[4]] + andhypers(p[4],1) + ands(p[5]+p[6]) + incq); } if (snc > 12) { incq = (tens[((p[2]*10)+p[3])] + andhypers((p[2]+p[3]+p[1]),5) + incq); } if (snc > 14) { incq = (andhypers(p[1],0) + ones[p[1]] + andhypers(p[1],1) + ands(p[2]+p[3]) + incq); } incq._chug(); if (ano) { incq = ("%lld".printf(n) + " (" + incq + ")"); } return incq; } void main () { string[] k = {}; k += "1000000001"; for (int64 i = 10; i < 10000000000000000; i *= 2) { if (i > 0) { k += "%lld".printf(i); } } //print("nums: \n{\n\t%s\n}\n",string.joinv(",\n\t",k)); string[] tens = maketens(); int64 mtts = GLib.get_monotonic_time(); for (int i = 0; i < k.length; i++) { int64 ii = int64.parse(k[i]); k[i] = inglicnums(ii,true,ref tens); } int64 mtte = GLib.get_monotonic_time(); print("nums+words: \n{\n\t%s\n}\n",string.joinv(",\n\t",k)); print("nums to text took %.4f seconds\n",(((double) (mtte - mtts)) / 1000000.0)); }
#+RESULTS:
nums+words: { 1000000001 (one billion and one), 10 (ten), 20 (twenty), 40 (fourty), 80 (eighty), 160 (one hundred and sixty), 320 (three hundred and twenty), 640 (six hundred and fourty), 1280 (one thousand two hundred and eighty), 2560 (two thousand five hundred and sixty), 5120 (five thousand one hundred and twenty), 10240 (ten thousand two hundred and fourty), 20480 (twenty thousand four hundred and eighty), 40960 (fourty thousand nine hundred and sixty), 81920 (eighty-one thousand nine hundred and twenty), 163840 (one hundred and sixty-three thousand eight hundred and fourty), 327680 (three hundred and twenty-seven thousand six hundred and eighty), 655360 (six hundred and fifty-five thousand three hundred and sixty), 1310720 (one million three hundred and ten thousand seven hundred and twenty), 2621440 (two million six hundred and twenty-one thousand four hundred and fourty), 5242880 (five million two hundred and fourty-two thousand eight hundred and eighty), 10485760 (ten million four hundred and eighty-five thousand seven hundred and sixty), 20971520 (twenty million nine hundred and seventy-one thousand five hundred and twenty), 41943040 (fourty-one million nine hundred and fourty-three thousand and fourty), 83886080 (eighty-three million eight hundred and eighty-six thousand and eighty), 167772160 (one hundred and sixty-seven million seven hundred and seventy-two thousand one hundred and sixty), 335544320 (three hundred and thirty-five million five hundred and fourty-four thousand three hundred and twenty), 671088640 (six hundred and seventy-one million and eighty-eight thousand six hundred and fourty), 1342177280 (one billion three hundred and fourty-two million one hundred and seventy-seven thousand two hundred and eighty), 2684354560 (two billion six hundred and eighty-four million three hundred and fifty-four thousand five hundred and sixty), 5368709120 (five billion three hundred and sixty-eight million seven hundred and nine thousand one hundred and twenty), 10737418240 (ten billion seven hundred and thirty-seven million four hundred and eighteen thousand two hundred and fourty), 21474836480 (twenty-one billion four hundred and seventy-four million eight hundred and thirty-six thousand four hundred and eighty), 42949672960 (fourty-two billion nine hundred and fourty-nine million six hundred and seventy-two thousand nine hundred and sixty), 85899345920 (eighty-five billion eight hundred and ninety-nine million three hundred and fourty-five thousand nine hundred and twenty), 171798691840 (one hundred and seventy-one billion seven hundred and ninety-eight million six hundred and ninety-one thousand eight hundred and fourty), 343597383680 (three hundred and fourty-three billion five hundred and ninety-seven million three hundred and eighty-three thousand six hundred and eighty), 687194767360 (six hundred and eighty-seven billion one hundred and ninety-four million seven hundred and sixty-seven thousand three hundred and sixty), 1374389534720 (one trillion three hundred and seventy-four billion three hundred and eighty-nine million five hundred and thirty-four thousand seven hundred and twenty), 2748779069440 (two trillion seven hundred and fourty-eight billion seven hundred and seventy-nine million and sixty-nine thousand four hundred and fourty), 5497558138880 (five trillion four hundred and ninety-seven billion five hundred and fifty-eight million one hundred and thirty-eight thousand eight hundred and eighty), 10995116277760 (ten trillion nine hundred and ninety-five billion one hundred and sixteen million two hundred and seventy-seven thousand seven hundred and sixty), 21990232555520 (twenty-one trillion nine hundred and ninety billion two hundred and thirty-two million five hundred and fifty-five thousand five hundred and twenty), 43980465111040 (fourty-three trillion nine hundred and eighty billion four hundred and sixty-five million one hundred and eleven thousand and fourty), 87960930222080 (eighty-seven trillion nine hundred and sixty billion nine hundred and thirty million two hundred and twenty-two thousand and eighty), 175921860444160 (one hundred and seventy-five trillion nine hundred and twenty-one billion eight hundred and sixty million four hundred and fourty-four thousand one hundred and sixty), 351843720888320 (three hundred and fifty-one trillion eight hundred and fourty-three billion seven hundred and twenty million eight hundred and eighty-eight thousand three hundred and twenty), 703687441776640 (seven hundred and three trillion six hundred and eighty-seven billion four hundred and fourty-one million seven hundred and seventy-six thousand six hundred and fourty), 1407374883553280 (squillions), 2814749767106560 (squillions), 5629499534213120 (squillions) } nums to text took 0.0002 seconds
JOINV:vala:
void main () { string[] s = {"1", "2", "3", "4", "5"}; print("joined = %s\n",string.joinv(";",s)); }
#+RESULTS:
joined = 1;2;3;4;5
LINESPERROW:vala:

Get a row offset given n lines per row, h height per line, y view hieght, o scroll offset...

int sumintarray (int[] a) { int s = 0; foreach (int v in a) { s += v; } return s; } string printintarray (int[] s, int f) { // 0 1 2 3 4 5 6 7 8 string[] ii = {"", "\"", "{", "[", "[", "| ", "", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n", ",\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%d".printf(s[i]) + dd[f]); } o = (o + "%d".printf(s[s.length - 1]) + oo[f]); return o; } void main() { // get row offset where there are multiple lines per row // by c.p.brown 2024 // // offset row only when it is fully scrolled out of view // // +--------------+------+ <---- -120px offset // | row 0 line 0 | 30 | // +--------------+------+ // | row 1 line 0 | 60 | DATA // | row 1 line 2 | 90 | // +--------------+------+ // | row 2 line 0 | 120 | <--- 4 lines above view // +-------------------------+-----+ // | | row 3 line 0 | 150 | | 30 | <-- row offset is +3 // | | row 3 line 1 | 180 | | 60 | // | | row 3 line 2 | 210 | | 90 | // | | row 3 line 3 | 240 | | 120 | // | | row 3 line 4 | 270 | | 150 | // | +--------------+------+ | | VIEW // | | row 4 line 0 | 300 | | 180 | // | | row 4 line 1 | 330 | | 210 | // | +--------------+------+ | | // | | row 5 line 0 | 360 | | 240 | // | | row 5 line 1 | 390 | | 270 | // | | row 5 line 2 | 420 | | 300 | <-- 10 lines in view // +-------------------------+-----+ // : : : // int[] rl = {1,2,1,5,2,3,1,6,2,3}; // lines per row int t = sumintarray(rl); // sum of lines double h = 30.0; // pixel height of line double y = 300.0; // pixel height of view double lv = y / h; // 10 lines in view double o = -120.0; // pixel scroll offset (from top) int lo = ((int) (o.abs() / h)); // 4 lines above view int[] lr = new int[t]; // row id per line int n = 0; for (int r = 0; r < rl.length; r++) { for (int l = 0; l < rl[r]; l++) { lr[n] = r; n += 1; } } print("lr is %s\n",printintarray(lr,2)); print("scroll offset is %d lines\n",lo); print("row offset is %d at line %d\n",lr[lo+1],(lo+1)); print("\n"); // scroll part-way through a cell... o = -133.0; lo = ((int) (o.abs() / h)); print("scroll offset at %.2f is %d lines\n",o,lo); print("row offset is %d at line %d\n",lr[lo+1],(lo+1)); o = -155.0; lo = ((int) (o.abs() / h)); print("scroll offset at %.2f is %d lines\n",o,lo); print("row offset is %d at line %d\n",lr[lo+1],(lo+1)); o = -162.0; lo = ((int) (o.abs() / h)); print("scroll offset at %.2f is %d lines\n",o,lo); print("row offset is %d at line %d\n",lr[lo+1],(lo+1)); }
#+RESULTS:
Compilation succeeded - 1 warning(s) lr is {0,1,1,2,3,3,3,3,3,4,4,5,5,5,6,7,7,7,7,7,7,8,8,9,9,9} scroll offset is 4 lines row offset is 3 at line 5 scroll offset at -133.00 is 4 lines row offset is 3 at line 5 scroll offset at -155.00 is 5 lines row offset is 3 at line 6 scroll offset at -162.00 is 5 lines row offset is 3 at line 6
LOADORGARTICLE:vala:

Load a top-level article from an org file.
(Double semi-colons inserted into results here to prevent orgception, which is not fun)

// load an orgmode heading and its sub-headings // by c.p.brown 2024 // remove double semicolons before use (I test these in orgmode itself) string loadinf (string t) { string doctitle = ("* " + t); string helpfile = "../../colyphyr/colyphyr_docs.org"; string oo = ""; GLib.File orgfile = GLib.File.new_for_path(helpfile); if (orgfile.query_exists() == true) { GLib.FileStream s = GLib.FileStream.open(helpfile,"r"); string l = s.read_line(); bool amreading = false; string tabs = ""; string aa = "**********"; while (l != null) { if (!amreading) { if (strcmp(l.substring(0,doctitle.length),doctitle) == 0) { amreading = true; tabs = ("%*s").printf(1," ").replace(" ","\t"); oo = (oo + ";;" + l + "\n"); l = s.read_line(); continue; } } else { int nind = 0; int ll = l.length; if (ll > 1) { if (strcmp(l.substring(0,2),"* ") == 0) { amreading = false; break; } } if (ll > 0) { if (strcmp(l.substring(0,1),"*") == 0) { for (int g = 2; g < 11; g++) { if (ll > g) { string ss = "%.*s ".printf(g,aa); if (strcmp(l.substring(0,(g+1)),ss) == 0) { nind = (g-1); } } } } } if (nind > 0) { tabs = ("%*s").printf(nind," ").replace(" ","\t"); oo = (oo + ";;" + tabs + l + "\n"); nind += 1; tabs = ("%*s").printf(nind," ").replace(" ","\t"); } else { oo = (oo + ";;" + tabs + l + "\n"); } } l = s.read_line(); } } return oo; } void main () { string s = "AUTOTYPE"; string a = loadinf(s); print(a); }
#+RESULTS:
;;* AUTOTYPE ;; ;; Attemps to determine type of a column by analyzing column content. ;; Autotype can be costly, use with caution. ;; ;; Min node inputs: 1 ;; Max node inputs: 1 ;; ;; ** Parameters ;; ;; 1. FIRST ;; Examine 1st row only. ;; ;; 2. LAST ;; Examine last row only ;; ;; 3. DELIMITER ;; String that delimits words in a list. ;; ;; 4. ALL ;; Examine all rows (slow). ;; ;; 5. RANDOM ;; Examine a random sample of rows. ;; ;; 1. ACCURACY (double) ;; Slider represents % of rows to sample. ;; Less samples = speed, more samples = accuracy. ;; ;; 6. ROW Num ;; Examine a singe ROW. ;; ;; 1. ROW (int) ;; Single row to examine. ;; ;; 7. ROW RANGE ;; Examine rows from FROM, to TO. ;; ;; 1. FROM (int) ;; First row of a range. ;; 2. TO (int) ;; Second row of a range. ;; ;; 8. NTH ROW ;; Examine every nth row from FROM by NTH. ;; ;; 1. FROM (int) ;; Start row. ;; 2. NTH (int) ;; Nth row to examine. ;; ;; ** Data ;; ;; 0. serarchby, str str, menu options: ;; 1. searchin, str int, what row number | row from ;; 2. searchto, str int, row to ;; 3. searchaccuracy, str double, samples as 0.0~1.0 ;; 4. searchdelimit, str str, list delimiter ;; ;; ** Examples ;; ;; rows=ALL, DELIMITER=" ", dat={ ;; "aa", "2024-06-20", "aa bb cc", "3", ;; "bb", "2024-07-10", "cc", "5", ;; "cc", "2024-09-29", "aa cc", "7" ;; } ;; = { ;; "header_0", "string", "24352624", "0", "0", ;; "header_1", "date", "37774721", "0", "0", ;; "header_2", "list", "98488123", "03749201","24352624", ;; "header_3", "int", "12628931", "0", "0" ;; }
LOOKAHEAD:vala:

Lookahead for an un-quoted, un-escaped character.

//lookahead for an unquoted, unescaped character // by c.p.brown 2024 void main() { GLib.Intl.setlocale(ALL,""); string s = """零: { find matching \{\} brace in «a \«string\» that contains }»}."""; unichar openchar = '{'; unichar closechar = '}'; unichar openquote = '«'; unichar closequote = '»'; unichar escapechar = '\\'; bool amquoted = false; string openstring = openchar.to_string(); int ii = s.index_of(openstring); int ci = -1; print(s + "\n"); print("has %d characers, and %d length\n",s.char_count(), s.length); int64 tts = GLib.get_monotonic_time(); if (ii != -1) { int i = 0; unichar c = 0; while (s.get_next_char(ref i, out c)) { if (c == escapechar) { s.get_next_char(ref i, out c); continue; } if (!amquoted && (c == openquote)) { amquoted = true; continue; } if (amquoted && (c == closequote)) { amquoted = false; continue; } if (!amquoted && (c == closechar)) { ci = i; break; } } } int64 tte = GLib.get_monotonic_time(); if (ci != -1) { print("lookahead took %d microseconds\n",((int) (tte - tts))); print("\'%s\' found at index %d\n",openchar.to_string(),ii); print("unquoted & unescaped \'%s\' found at index %d\n",closechar.to_string(),ci); print("captured region is \n%s\n",s.substring(ii,ci-ii)); } }
#+RESULTS:
零: { find matching \{\} brace in «a \«string\» that contains }»}. has 65 characers, and 71 length lookahead took 1 microseconds '{' found at index 5 unquoted & unescaped '}' found at index 70 captured region is { find matching \{\} brace in «a \«string\» that contains }»}
MEDIAN:vala:
// get median int // by c.p.brown 2024 void snortint (int[] l) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = l.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; j -= g; } l[j] = t; } } } double getmedianint (int[] n) { int l = n.length; double m = -1.0; if ((l & 1) == 0) { int aa = ((int) (l * 0.5)) - 1; int bb = aa + 1; print("left = %d, right = %d\n",n[aa],n[bb]); m = ((n[aa] + n[bb]) * 0.5); print("median is %.2f\n\n",m); } else { int aa = ((int) (l * 0.5)); m = (double) n[aa]; print("median is %.2f\n\n",m); } return m; } void main() { int[] n = {1, 2, 3, 4, 5, 6, 7, 8, 9}; print("n is {"); foreach (int i in n) { print("%d,",i); } print("}\n"); double m = getmedianint (n); n = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}; print("n is {"); foreach (int i in n) { print("%d,",i); } print("}\n"); m = getmedianint (n); n = {77, 4, 1, 234, 7, 84, 80, 98, 2, 101}; print("n is {"); foreach (int i in n) { print("%d,",i); } print("}\n"); snortint(n); print("sorted n is {"); foreach (int i in n) { print("%d,",i); } print("}\n"); m = getmedianint (n); }
#+RESULTS:
n is {1,2,3,4,5,6,7,8,9,} median is 5.00 n is {1,2,3,4,5,6,7,8,9,10,} left = 5, right = 6 median is 5.50 n is {77,4,1,234,7,84,80,98,2,101,} sorted n is {1,2,4,7,77,80,84,98,101,234,} left = 77, right = 80 median is 78.50
MASKARRAY:wip::vala:

Mask arrays using cell masks or bounding regions.
TODO:
  • colmask mode
  • rowmask mode
  • render functon for boxmask, colmask and rowmask, before unrolling to local temp mask
  • more dimension tests
  • clean up cellmask cropping
  • mask removal functions

// masking string arrays using a cellmask or bounding region // optionally resize data dimensions // by c.p.brown 2024 // mask format is is alternating indices: // current cell index (after crop, xform), original source cell index // eg: // // source data indices with 3 cols: // 0 1 2 // 3 4 5 // 6 7 8 // // input mask with 3 cols: // - - - // - 4 - // - 7 8 // dimension mode: CROP // // cropped mask: // 4 - // 7 8 // // temporary local mask format: // // new dat index cols // | | | | // mask = { 0, 4, 2, 7, 3, 8 }; msz = 2 // | | | // old dat index // // to mask an operation: // for (int i = 0; i < msk.length; i+=2) { // dat[msk[i]] = mytask(ops[sop].dat[msk[i+1]]); // } // // to multithread masked tasks // // debug indentation item count total packets nth item // \ | | / // packet[] pak = packseq(0,msk,msk.length,true,maxpackets,0,2); // | | | // items use items offset item // // create threadpool, assign packets to worker object // // int[] threadcheck = new int[pak.length]; // for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } // ThreadPool<workerobject> tp = new ThreadPool<workerobject>.with_owned_data( // (workerobject) => {workerobject.run(ref threadcheck);},threadcount,false // ); // for (int i = 0; i < pak.length; i++) { // tp.add( new workerobject(i,pak[i].items) ); // } // int lastindex = threadckeck.length - 1; // while (threadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } // int maxintarray (int[] a) { int m = -999999999; int o = -1; for (int x = 0; x < a.length; x++) { if (a[x] > m) { m = a[x]; o = x; } } return o; } uint maxuintarray (uint[] a) { uint m = 0; uint o = 0; for (uint x = 0; x < a.length; x++) { if (a[x] > m) { m = a[x]; o = x; } } return o; } string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } string printuintarray (uint[] s, uint f, uint b) { string[] ii = {"", "\"", "{", "[", "[", "| ", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", ""}; string o = ii[f]; bool bitshift = (b > 0); for (uint i = 0; i < (s.length - 1); i++) { if (bitshift) { o = (o + "%u".printf((s[i] >> b)) + dd[f]); } else { o = (o + "%u".printf(s[i]) + dd[f]); } } if (bitshift) { o = (o + "%u".printf((s[s.length - 1] >> b)) + oo[f]); } else { o = (o + "%u".printf(s[s.length - 1]) + oo[f]); } return o; } class dop : Object { public string[] dat; // string data, as a 1d array public uint dsz; // data cols private uint[] msk; // cellmask private uint msz; // cellmask cols private uint[] bbx; // box mask public void datset (string[] d, uint c) { dsz = c; dat.resize(0); for (uint i = 0; i < d.length; i++) { StringBuilder cel = new StringBuilder(""); cel.append(d[i]); dat += cel.str; } } public void mskset (uint[] m, uint c) { msz = c; msk.resize(0); for (uint i = 0; i < m.length; i++) { msk += m[i]; } } public void bbxset (uint[] b) { bbx.resize(0); for (int i = 0; i < b.length; i++) { bbx += b[i]; } } public string rendermask () { string o = ""; if ((msk.length > 0) && (msz > 0)) { uint moplastcell = maxuintarray(msk); uint moplen = msk[moplastcell]; uint moplastcol = ((moplastcell + 1) % msz); moplen += (msz - moplastcol); uint moprowcount = (uint) ((double) moplen / (double) msz); uint n = 0; for (uint i = 0; i < moplen; i++) { if (n < msk.length) { if (i == msk[n]) { o = (o + " ██ "); n += 1; } else { o = (o + " %02u ".printf(i)); } } else { o = (o + " %02u ".printf(i)); } if (((i + 1) % msz) == 0) { o = (o + "\n"); } } } return o; } public string datreport (uint ind, bool h, bool m, int sp) { string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string spc = " "; int sop = sp; bool domask = m; string dlm = " | "; uint[] cw = new uint[dsz]; for (int i = 0; i < cw.length; i++) { cw[i] = 0; } if (h) { for (uint i = 0; i < cw.length; i++) { string hh = "col %u".printf(i); cw[i] = hh.char_count(); } } string o = ""; string ln = ""; // get counts 1st for (uint i = 0; i < dat.length; i++) { uint c = (i % dsz); int cc = 0; if (dat[i] != null) { cc = dat[i].char_count(); if (cc > 0) { int nwlat = dat[i].index_of("\n"); if (nwlat != -1) { cc = (nwlat + 3); } } } cw[c] = uint.max(cw[c],cc); } // print header if (h) { for (uint i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } string hh = "col %u".printf(i); //int cc = hh.char_count(); o = "%s%s%-*s".printf(o,dlm,cw[i],hh); if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } // print line for (uint i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s|-".printf(tabs); } ln = "%s%s%-*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-|\n".printf(ln); } } o = "%s%s".printf(o,ln); } // make rows int n = 0; for (uint i = 0; i < dat.length; i++) { uint c = (i % dsz); string datpart = " "; bool trunc = false; int cc = 0; if (dat[i] != null) { cc = dat[i].char_count(); if (cc > 0) { datpart = dat[i]; int nwlat = dat[i].index_of("\n"); if (nwlat != -1) { trunc = true; datpart = (dat[i].substring(0,nwlat) + "..."); cc = (nwlat + 3); } } else { cc = int.max(cc,1); } } dlm = " | "; if (c == 0) { dlm = "%s| ".printf(tabs); } int padlen = (((int) cw[c]) - cc); padlen = int.max(padlen,0); datpart = "%s%.*s".printf(datpart,padlen,spc); if (domask) { bool ismasked = ((n < msk.length) && (msk[n] == i)); if (ismasked) { n += 2; datpart = "\x1b[48;5;5m%s\x1b[0m".printf(datpart); } } o = "%s%s%s".printf(o,dlm,datpart); if (c == (dsz - 1)) { o = "%s |\n".printf(o); } } cw = null; return o; } public void maskmemydata (int sp, int mp, string dm, bool cm, bool fm) { msk.resize(0); msz = 0; bbx.resize(0); int mop = mp; // index of op containing mask int sop = sp; // index of op containing source data bool cellmode = cm; bool dofocus = fm; uint moplen = ops[mop].msk.length; uint soplen = ops[sop].dat.length; uint mopcolcount = ops[mop].msz; uint sopcolcount = ops[sop].dsz; uint soprowcount = (uint) ((double) soplen / (double) sopcolcount); bool domin = (strcmp(dm,"MINIMUM") == 0); bool domax = (strcmp(dm,"MAXIMUM") == 0); bool domatch = (strcmp(dm,"MASK") == 0); bool docrop = (strcmp(dm,"CROP") == 0); bool dosource = (strcmp(dm,"SOURCE") == 0); uint resultcolcount = sopcolcount; uint resultrowcount = soprowcount; uint mopfirstcol = 0; uint mopfirstrow = 0; uint moplastcol = resultcolcount - 1; uint moplastrow = resultrowcount - 1; uint newdatlen = ops[sop].dat.length; if (cellmode) { // mask bounds uint moplastcell = maxuintarray(ops[mop].msk); moplen = ops[mop].msk[moplastcell]; moplastcol = ((moplastcell + 1) % mopcolcount); moplen += (mopcolcount - moplastcol); uint moprowcount = (uint) ((double) moplen / (double) mopcolcount); // crop bounds uint mincolcount = uint.min(sopcolcount,mopcolcount); uint minrowcount = uint.min(soprowcount,moprowcount); uint maxcolcount = uint.max(sopcolcount,mopcolcount); uint maxrowcount = uint.max(soprowcount,moprowcount); if (docrop) { uint cropminx = 1000000000; uint cropminy = 1000000000; uint cropmaxx = 0; uint cropmaxy = 0; int n = 0; for (uint i = 0; i < moplen; i++) { uint c = i % mopcolcount; uint r = (uint) ((double) i / (double) mopcolcount); if ((c < mincolcount) && (r < minrowcount)) { if (i == ops[mop].msk[n]) { cropminx = uint.min(c,cropminx); cropminy = uint.min(r,cropminy); n += 1; } } } n = 0; for (uint i = 0; i < moplen; i++) { uint c = i % mopcolcount; uint r = (uint) ((double) i / (double) mopcolcount); if ((c < mincolcount) && (r < minrowcount)) { if (i == ops[mop].msk[n]) { cropmaxx = uint.max(c,cropmaxx); cropmaxy = uint.max(r,cropmaxy); n += 1; } } } mincolcount = ((cropmaxx + 1) - cropminx); minrowcount = ((cropmaxy + 1) - cropminy); // get cropmax again, now that we have the offsets n = 0; for (uint i = 0; i < moplen; i++) { uint c = i % mopcolcount; uint r = (uint) ((double) i / (double) mopcolcount); if (((c - cropminx) < mincolcount) && ((r - cropminy) < minrowcount)) { if (i == ops[mop].msk[n]) { cropmaxx = uint.max(c,cropmaxx); cropmaxy = uint.max(r,cropmaxy); n += 1; } } } if (cropmaxx == 0) { cropmaxx = mincolcount - 1; } if (cropmaxy == 0) { cropmaxy = minrowcount - 1; } resultcolcount = (cropmaxx + 1) - cropminx; resultrowcount = (cropmaxy + 1) - cropminy; mopfirstcol = cropminx; mopfirstrow = cropminy; moplastcol = cropmaxx; moplastrow = cropmaxy; } if (domin) { resultcolcount = mincolcount; resultrowcount = minrowcount; mopfirstcol = 0; mopfirstrow = 0; moplastcol = resultcolcount - 1; moplastrow = resultrowcount - 1; } if (domax) { resultcolcount = maxcolcount; resultrowcount = maxrowcount; mopfirstcol = 0; mopfirstrow = 0; moplastcol = resultcolcount - 1; moplastrow = resultrowcount - 1; } if (dosource) { resultcolcount = sopcolcount; resultrowcount = soprowcount; mopfirstcol = 0; mopfirstrow = 0; moplastcol = resultcolcount - 1; moplastrow = resultrowcount - 1; } if (domatch) { resultcolcount = mopcolcount; resultrowcount = moprowcount; mopfirstcol = 0; mopfirstrow = 0; moplastcol = resultcolcount - 1; moplastrow = resultrowcount - 1; } msk.resize(0); msz = resultcolcount; newdatlen = (resultrowcount * resultcolcount); dsz = msz; dat.resize((int) newdatlen); for (int i = 0; i < dat.length; i++) { dat[i] = ""; } int n = 0; mincolcount = uint.min(sopcolcount,moprowcount); minrowcount = uint.min(soprowcount,moprowcount); maxcolcount = uint.max(sopcolcount,mopcolcount); maxrowcount = uint.max(soprowcount,moprowcount); for (uint moprow = mopfirstrow; moprow <= moplastrow; moprow++) { for (uint mopcol = mopfirstcol; mopcol <= moplastcol; mopcol += 1) { uint thiscellindex = ((moprow - mopfirstrow) * resultcolcount) + (mopcol - mopfirstcol); uint sopcellindex = ops[sop].dat.length; uint mopcellindex = (moprow * mopcolcount) + mopcol; if ((moprow < soprowcount) && (mopcol < sopcolcount)) { sopcellindex = (moprow * sopcolcount) + mopcol; if ((moprow < moprowcount) && (mopcol < mopcolcount)) { if (mopcellindex == ops[mop].msk[n]) { msk += thiscellindex; msk += sopcellindex; n += 1; } } dat[thiscellindex] = ops[sop].dat[sopcellindex]; } } } } else { if (ops[mop].bbx.length == 4) { uint moprowcount = (uint) ((double) moplen / (double) mopcolcount); mopcolcount = (ops[mop].bbx[2] + 1); moprowcount = (ops[mop].bbx[3] + 1); if (domax) { mopcolcount = (ops[mop].bbx[2] + 1); moprowcount = (ops[mop].bbx[3] + 1); resultcolcount = uint.max(sopcolcount,(ops[mop].bbx[2] + 1)); resultrowcount = uint.max(soprowcount,(ops[mop].bbx[3] + 1)); mopfirstcol = 0; mopfirstrow = 0; moplastcol = resultcolcount - 1; moplastrow = resultrowcount - 1; } if (domin) { mopcolcount = (ops[mop].bbx[2] + 1); moprowcount = (ops[mop].bbx[3] + 1); resultcolcount = uint.min(sopcolcount,mopcolcount); resultrowcount = uint.min(soprowcount,moprowcount); mopfirstcol = 0; mopfirstrow = 0; moplastcol = resultcolcount - 1; moplastrow = resultrowcount - 1; } if (docrop) { mopcolcount = (ops[mop].bbx[2] + 1); moprowcount = (ops[mop].bbx[3] + 1); resultcolcount = uint.min(sopcolcount,mopcolcount) - ops[mop].bbx[0]; resultrowcount = uint.min(soprowcount,moprowcount) - ops[mop].bbx[1]; mopfirstcol = ops[mop].bbx[0]; mopfirstrow = ops[mop].bbx[1]; moplastcol = ops[mop].bbx[2]; moplastrow = ops[mop].bbx[3]; } if (domatch) { mopcolcount = (ops[mop].bbx[2] + 1); moprowcount = (ops[mop].bbx[3] + 1); resultcolcount = ((ops[mop].bbx[2] + 1) - ops[mop].bbx[0]); resultrowcount = ((ops[mop].bbx[3] + 1) - ops[mop].bbx[1]); mopfirstcol = ops[mop].bbx[0]; mopfirstrow = ops[mop].bbx[1]; moplastcol = ops[mop].bbx[2]; moplastrow = ops[mop].bbx[3]; } msk.resize(0);// = new uint[(resultcolcount * resultrowcount)]; newdatlen = (resultcolcount * resultrowcount); msz = resultcolcount; dsz = msz; dat.resize((int) newdatlen); for (int i = 0; i < dat.length; i++) { dat[i] = ""; } int n = 0; for (uint mopcol = mopfirstcol; mopcol <= moplastcol; mopcol++) { for (uint moprow = mopfirstrow; moprow <= moplastrow; moprow++) { uint thiscellindex = ((moprow - mopfirstrow) * resultcolcount) + (mopcol - mopfirstcol); uint mopcellindex = (moprow * mopcolcount) + mopcol; uint sopcellindex = ops[sop].dat.length; if ((mopcol < sopcolcount) && (moprow < soprowcount)) { sopcellindex = (moprow * sopcolcount) + mopcol; //if (domatch) { print("cell %u points to src cell %u %s\n",thiscellindex,sopcellindex,ops[sop].dat[sopcellindex]); } if ( (mopcol >= ops[mop].bbx[0]) && (mopcol <= ops[mop].bbx[2]) && (moprow >= ops[mop].bbx[1]) && (moprow <= ops[mop].bbx[3]) ) { msk += thiscellindex; msk += sopcellindex; n += 1; } dat[thiscellindex] = ops[sop].dat[sopcellindex]; } } } } } } public dop () { msk = {}; dat = {}; bbx = {}; } } // container for data objects dop[] ops; void main() { GLib.Intl.setlocale(ALL,""); string p = "█"; // 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 string[] d = {"A","B","C","X","Y\n","Z","0","1","2","<","=",">","&","!","^","\\",":","/"}; // \ \ \ / / // 00 01 02 03 04 05 06 | // \\ // | 00 // 07 08 09 10 11 12 13 | // \\ // | 01 // 14 15 16 17 18 19 20 | // \\ // | 02 // 21 22 23 24 25 26 27 | // xx | 03 // 28 29 30 31 32 33 34 | // // \\ | 04 // 35 36 37 38 39 40 41 | // // \\ | 05 // 42 43 44 45 46 47 48 | // // \\ | 06 // source ops += new dop(); ops[0].datset(d,3); print("data is:\n%s\n",ops[0].datreport(1,true,false,0)); // mask ops += new dop(); //uint[] cellmask = {0,6,8,12,16,18,24,30,32,36,40,42,48}; // cross 7 cols //uint[] cellmask = {8,16,24,32,40,48}; // partial backslash 7 cols uint[] cellmask = {2,3,5,6,7}; // reverse c 2 cols ops[1].mskset(cellmask,2); print("mask is:\n%s",ops[1].rendermask()); uint[] bbox = {1,1,4,1}; ops[1].bbxset(bbox); // this op ops += new dop(); // src mask hide unmasked data // \ / | ops[2].maskmemydata(0,1,"SOURCE",true,false); // | | // size cellmask // otherwise // boxmask print("\ncall masked data with source dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"MAXIMUM",true,false); print("\ncell masked data with max dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"MINIMUM",true,false); print("\ncell masked data with min dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"MASK",true,false); print("\ncell masked data with mask dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"CROP",true,false); print("\ncell masked data with cropped dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"SOURCE",false,false); print("\nbox masked data with source dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"MAXIMUM",false,false); print("\nbox masked data with maximum dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"MINIMUM",false,false); print("\nbox masked data with minimum dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"CROP",false,false); print("\nbox masked data with cropped dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); ops[2].maskmemydata(0,1,"MASK",false,false); print("\nbox masked data with mask dimensions is:\n%s\n",ops[2].datreport(1,false,true,0)); }
#+RESULTS:
Compilation succeeded - 8 warning(s) data is: | col 0 | col 1 | col 2 | |-------+-------+-------| | A | B | C | | X | Y... | Z | | 0 | 1 | 2 | | < | = | > | | & | ! | ^ | | \ | : | / | mask is: 00 01 ██ ██ 04 ██ ██ ██ call masked data with source dimensions is: | A | B | C | | X | Y... | Z | | 0 | 1 | 2 | | < | = | > | | & | ! | ^ | | \ | : | / | cell masked data with max dimensions is: | A | B | C | | X | Y... | Z | | 0 | 1 | 2 | | < | = | > | | & | ! | ^ | | \ | : | / | cell masked data with min dimensions is: | A | B | | X | Y... | | 0 | 1 | | < | = | cell masked data with mask dimensions is: | A | B | | X | Y... | | 0 | 1 | | < | = | cell masked data with cropped dimensions is: | X | Y... | | 0 | 1 | | < | = | box masked data with source dimensions is: | A | B | C | | X | Y... | Z | | 0 | 1 | 2 | | < | = | > | | & | ! | ^ | | \ | : | / | box masked data with maximum dimensions is: | A | B | C | | | | X | Y... | Z | | | | 0 | 1 | 2 | | | | < | = | > | | | | & | ! | ^ | | | | \ | : | / | | | box masked data with minimum dimensions is: | A | B | C | | X | Y... | Z | box masked data with cropped dimensions is: | Y... | Z | box masked data with mask dimensions is: | Y... | Z | | |
MODULO:vala:

Misc modulo sanity-checks go here...

int imod (int l, int r) { if (l >= 0) { return (l % r); } if (l >= -r) { return (l + r); } return ((l % r) + r) % r; } void main() { for (int i = 0; i < 32; i++) { if ((i % 9) == 0) { print("\n"); } print("ROW %d (%d / %d)\n",(i / 9),1,9); print("COL %d (%d %% %d)\n",(i % 9),i,9); print(""); } int[] nums = {}; for (int i = 0; i < 10; i++) { nums += i; } int l = nums.length; print("\ncycle through an array of 10 items, from 0 to 9 using i %% length; (10)...\n"); for (int i = 3; i < 12; i++) { print("nums[%d] is %d\n",(i % l),nums[(i % l)]); } print("\n cycle through several lots of 1 to 12...\n"); for (int i = 0; i < 30; i++) { print("((%d %% 12) + 1) is %d\n",i,((i % 12) + 1)); } print("\nsubtract 1 from a month, wrapping from 1 to 12...\n"); for (int i = 1; i < 13; i++) { print("month (%d-1) is %d\n",i,(i-1)); print("month (((%d-2)%%12)+1) is %d\n",i,(imod((i-2),12)+1)); } print("\nis it a leap year?\n"); print("1600 % 100 is %d\n",(1600 % 100)); print("1600 % 400 is %d\n",(1600 % 400)); print("1600 % 4 is %d\n",(1600 % 4)); print("1700 % 100 is %d\n",(1700 % 100)); print("1700 % 400 is %d\n",(1700 % 400)); print("1700 % 4 is %d\n",(1700 % 4)); print("1800 % 100 is %d\n",(1800 % 100)); print("1800 % 400 is %d\n",(1800 % 400)); print("1800 % 4 is %d\n",(1800 % 4)); print("1900 % 100 is %d\n",(1900 % 100)); print("1900 % 400 is %d\n",(1900 % 400)); print("1900 % 4 is %d\n",(1900 % 4)); print("2000 % 100 is %d\n",(2000 % 100)); print("2000 % 400 is %d\n",(2000 % 400)); print("2000 % 4 is %d\n",(2000 % 4)); print("2024 % 100 is %d\n",(2024 % 100)); print("2024 % 400 is %d\n",(2024 % 400)); print("2024 % 4 is %d\n",(2024 % 4)); print("2025 % 100 is %d\n",(2025 % 100)); print("2025 % 400 is %d\n",(2025 % 400)); print("2025 % 4 is %d\n",(2025 % 4)); print("2100 % 100 is %d\n",(2100 % 100)); print("2100 % 400 is %d\n",(2100 % 400)); print("2100 % 4 is %d\n",(2100 % 4)); }
#+RESULTS:
ROW 0 (1 / 9) COL 0 (0 % 9) ROW 0 (1 / 9) COL 1 (1 % 9) ROW 0 (1 / 9) COL 2 (2 % 9) ROW 0 (1 / 9) COL 3 (3 % 9) ROW 0 (1 / 9) COL 4 (4 % 9) ROW 0 (1 / 9) COL 5 (5 % 9) ROW 0 (1 / 9) COL 6 (6 % 9) ROW 0 (1 / 9) COL 7 (7 % 9) ROW 0 (1 / 9) COL 8 (8 % 9) ROW 1 (1 / 9) COL 0 (9 % 9) ROW 1 (1 / 9) COL 1 (10 % 9) ROW 1 (1 / 9) COL 2 (11 % 9) ROW 1 (1 / 9) COL 3 (12 % 9) ROW 1 (1 / 9) COL 4 (13 % 9) ROW 1 (1 / 9) COL 5 (14 % 9) ROW 1 (1 / 9) COL 6 (15 % 9) ROW 1 (1 / 9) COL 7 (16 % 9) ROW 1 (1 / 9) COL 8 (17 % 9) ROW 2 (1 / 9) COL 0 (18 % 9) ROW 2 (1 / 9) COL 1 (19 % 9) ROW 2 (1 / 9) COL 2 (20 % 9) ROW 2 (1 / 9) COL 3 (21 % 9) ROW 2 (1 / 9) COL 4 (22 % 9) ROW 2 (1 / 9) COL 5 (23 % 9) ROW 2 (1 / 9) COL 6 (24 % 9) ROW 2 (1 / 9) COL 7 (25 % 9) ROW 2 (1 / 9) COL 8 (26 % 9) ROW 3 (1 / 9) COL 0 (27 % 9) ROW 3 (1 / 9) COL 1 (28 % 9) ROW 3 (1 / 9) COL 2 (29 % 9) ROW 3 (1 / 9) COL 3 (30 % 9) ROW 3 (1 / 9) COL 4 (31 % 9) cycle through an array of 10 items, from 0 to 9 using i % length; (10)... nums[3] is 3 nums[4] is 4 nums[5] is 5 nums[6] is 6 nums[7] is 7 nums[8] is 8 nums[9] is 9 nums[0] is 0 nums[1] is 1 cycle through several lots of 1 to 12... ((0 % 12) + 1) is 1 ((1 % 12) + 1) is 2 ((2 % 12) + 1) is 3 ((3 % 12) + 1) is 4 ((4 % 12) + 1) is 5 ((5 % 12) + 1) is 6 ((6 % 12) + 1) is 7 ((7 % 12) + 1) is 8 ((8 % 12) + 1) is 9 ((9 % 12) + 1) is 10 ((10 % 12) + 1) is 11 ((11 % 12) + 1) is 12 ((12 % 12) + 1) is 1 ((13 % 12) + 1) is 2 ((14 % 12) + 1) is 3 ((15 % 12) + 1) is 4 ((16 % 12) + 1) is 5 ((17 % 12) + 1) is 6 ((18 % 12) + 1) is 7 ((19 % 12) + 1) is 8 ((20 % 12) + 1) is 9 ((21 % 12) + 1) is 10 ((22 % 12) + 1) is 11 ((23 % 12) + 1) is 12 ((24 % 12) + 1) is 1 ((25 % 12) + 1) is 2 ((26 % 12) + 1) is 3 ((27 % 12) + 1) is 4 ((28 % 12) + 1) is 5 ((29 % 12) + 1) is 6 subtract 1 from a month, wrapping from 1 to 12... month (1-1) is 0 month (((1-2)%12)+1) is 12 month (2-1) is 1 month (((2-2)%12)+1) is 1 month (3-1) is 2 month (((3-2)%12)+1) is 2 month (4-1) is 3 month (((4-2)%12)+1) is 3 month (5-1) is 4 month (((5-2)%12)+1) is 4 month (6-1) is 5 month (((6-2)%12)+1) is 5 month (7-1) is 6 month (((7-2)%12)+1) is 6 month (8-1) is 7 month (((8-2)%12)+1) is 7 month (9-1) is 8 month (((9-2)%12)+1) is 8 month (10-1) is 9 month (((10-2)%12)+1) is 9 month (11-1) is 10 month (((11-2)%12)+1) is 10 month (12-1) is 11 month (((12-2)%12)+1) is 11 is it a leap year? 1600 % 100 is 0 1600 % 400 is 0 1600 % 4 is 0 1700 % 100 is 0 1700 % 400 is 100 1700 % 4 is 0 1800 % 100 is 0 1800 % 400 is 200 1800 % 4 is 0 1900 % 100 is 0 1900 % 400 is 300 1900 % 4 is 0 2000 % 100 is 0 2000 % 400 is 0 2000 % 4 is 0 2024 % 100 is 24 2024 % 400 is 24 2024 % 4 is 0 2025 % 100 is 25 2025 % 400 is 25 2025 % 4 is 1 2100 % 100 is 0 2100 % 400 is 100 2100 % 4 is 0
NESTEDLOGIC:vala:


Group items of a comparison list using an indent value, to emulate things like:
true AND true OR (false OR true) AND true

// cumulative grouped tests // by c.p.brown 2024 struct packet { int[] items; } packet[] clusteredints (int[] n) { packet[] o = {}; packet p = packet(); p.items += 0; o += p; for (int i = 1; i < n.length; i++) { if ((n[i] - n[i-1]) != 0) { packet b = packet(); b.items += i; o += b; } else { o[o.length - 1].items += i; } } return o; } string printclusters (packet[] b) { string o = ""; for (int c = 0; c < b.length; c++) { o = "%sblock %d has %d members ".printf(o,c,b[c].items.length); o = "%s\t{".printf(o); for (int i = 0; i < b[c].items.length; i++) { if (i < (b[c].items.length - 1)) { o = "%s%d,".printf(o,b[c].items[i]); } else { o = "%s%d}\n".printf(o,b[c].items[i]); } } } return o; } void main () { // +---+--------------------+-------+---------------+---------------------+ // | 0 | B == B = true | OR | true = true | true OR | // +---+--------------------+-------+---------------+ | // | 1 | B == A = false | AND | false AND | | // +---+--------------------+-------+ | false = true, AND | // | 1 | B == C = false | AND | false = false | | // +---+--------------------+-------+---------------+ | // | 0 | B == B = true | (AND) | true = true | true = true | // +---+--------------------+-------+---------------+---------------------+ // | d | 1. eval each rule | c | 2. compare | 3. compare clusters | // | | | | per cluster | | // +---+--------------------+-------+---------------+---------------------+ // // for round 3, use last comparison (c) of the cluster, apply to next cluster // omit comparison from last rule as nothing follows it string[] t = {"B","A","C","B"}; string[] c = {"OR","AND","AND","AND"}; int[] d = {1,0,0,1}; string q = "B"; packet[] qq = clusteredints(d); print(printclusters(qq)); bool[] j = new bool[t.length]; bool[] r = new bool[qq.length]; for (int i = 0; i < r.length; i++) { r[i] = false; } //r[0] = true; if (false && false) { print("double negatives are true in Vala\n"); } // 1. stash tests... for (int i = 0; i < j.length; i++){ j[i] = (t[i] == q); if (j[i]) { print("[%d] %s == %s is true\n",i,q,t[i]); } else { print("[%d] %s == %s is false\n",i,q,t[i]); } } // 2. cumulative test, per packet for (int k = 0; k < qq.length; k++) { bool lc = j[qq[k].items[0]]; for (int i = 0; i < qq[k].items.length; i++) { print("block %d, index %d\n",k,qq[k].items[i]); int lcx = qq[k].items[i]; if (strcmp(c[qq[k].items[i-1]],"AND") == 0) { string lcs = "false"; if (lc) { lcs = "true"; } string jcs = "false"; if (j[lcx]) { jcs = "true"; } if (i > 0) { lc = (j[lcx] && lc); if (lc) { print("[%d][%d] (rule result) %s && (lastcond) %s is true\n",k,lcx,jcs,lcs); } else { print("[%d][%d] (rule result) %s && (lastcond) %s is false\n",k,lcx,jcs,lcs); } } else { if (lc) { print("[%d][%d] (rule result) %s is true\n",k,lcx,lcs); } else { print("[%d][%d] (rule result) %s is false\n",k,lcx,lcs); } } } else { if (strcmp(c[qq[k].items[i-1]],"OR") == 0) { string lcs = "false"; if (lc) { lcs = "true"; } string jcs = "false"; if (j[lcx]) { jcs = "true"; } if (i > 0) { lc = (j[lcx] && lc); if (lc) { print("[%d][%d] (rule result) %s || (lastcond) %s is true\n",k,lcx,jcs,lcs); } else { print("[%d][%d] (rule result) %s || (lastcond) %s is false\n",k,lcx,jcs,lcs); } } else { if (lc) { print("[%d][%d] (rule result) %s is true\n",k,lcx,lcs); } else { print("[%d][%d] (rule result) %s is false\n",k,lcx,lcs); } } } } } r[k] = lc; } // 3. cumulative test bool lc = r[0]; for (int k = 1; k < qq.length; k++) { int lcx = qq[k-1].items[(qq[k-1].items.length - 1)]; if (strcmp(c[lcx],"AND") == 0) { string lcs = "false"; if (lc) { lcs = "true"; } string rks = "false"; if (r[k]) { rks = "true"; } lc = r[k] && lc; if (lc) { print("[%d] %s && %s is true\n",k,rks,lcs); } else { print("[%d] %s && %s is false\n",k,rks,lcs); } } else { if (strcmp(c[lcx],"OR") == 0) { string lcs = "false"; if (lc) { lcs = "true"; } string rks = "false"; if (r[k]) { rks = "true"; } lc = r[k] || lc; if (lc) { print("[%d] %s || %s is true\n",k,rks,lcs); } else { print("[%d] %s || %s is false\n",k,rks,lcs); } } } } if (lc) { print("result is true\n"); } else { print("result is false\n"); } }
#+RESULTS:
Compilation succeeded - 1 warning(s) block 0 has 1 members {0} block 1 has 2 members {1,2} block 2 has 1 members {3} [0] B == B is true [1] B == A is false [2] B == C is false [3] B == B is true block 0, index 0 [0][0] (rule result) true is true block 1, index 1 [1][1] (rule result) false is false block 1, index 2 [1][2] (rule result) false && (lastcond) false is false block 2, index 3 [2][3] (rule result) true is true [1] false || true is true [2] true && true is true result is true
NTHROW:vala:

Check math for row nth + offset in a 1D array.
Used to reduce iteration in certian cases.

void main() { string[] t = { "ROW1_COL0", "ROW1_COL1", "ROW1_COL2", "ROW2_COL0", "ROW2_COL1", "ROW2_COL2", "ROW3_COL0", "ROW3_COL1", "ROW3_COL2", "ROW4_COL0", "ROW4_COL1", "ROW4_COL2", "ROW5_COL0", "ROW5_COL1", "ROW5_COL2", "ROW6_COL0", "ROW6_COL1", "ROW6_COL2", "ROW7_COL0", "ROW7_COL1", "ROW7_COL2", "ROW8_COL0", "ROW8_COL1", "ROW8_COL2", "ROW9_COL0", "ROW9_COL1", "ROW9_COL2" }; int nth = 2; int csz = 3; print("array has %d cells per row\n",csz); int ofs = 2; int ixs = ofs * csz; print("offset index is %d\n",ixs); int ixi = nth * csz; print("nth %d rows skips %d cells\n\n",nth,ixi); int c = 0; for (int i = ixs; i < t.length; i++) { print("[%02d] %s\n",i,t[i]); c += 1; if (c == csz) { i += ixi; c = 0; print("...\n"); } } }
#+RESULTS:
array has 3 cells per row offset index is 6 nth 2 rows skips 6 cells [06] ROW3_COL0 [07] ROW3_COL1 [08] ROW3_COL2 ... [15] ROW6_COL0 [16] ROW6_COL1 [17] ROW6_COL2 ... [24] ROW9_COL0 [25] ROW9_COL1 [26] ROW9_COL2 ...
OPTYPE:vala:

Run object functions without knowing the object type.

// test abstract classes & overrides // by c.p.brown 2024 void snortintsixtyfour (int64[] l) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = l.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { int64 t = l[i]; int j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; j -= g; } l[j] = t; } } } double getmedianintsixtyfour (int64[] n) { int l = n.length; double m = -1.0; if ((l & 1) == 0) { int aa = ((int) (l * 0.5)) - 1; int bb = aa + 1; m = (double) ((n[aa] + n[bb]) * 0.5); } else { int aa = ((int) (l * 0.5)); m = (double) n[aa]; } return m; } public abstract class xop : Object { public int myindex; public string outputstring; public abstract void vrun (); } public class xopconcat : xop { private weak string[] sref; public xopconcat (int x, string[] s) { myindex = x; sref = s; } public override void vrun () { foreach (string l in sref) { outputstring = (outputstring + l); } } } public class xoplist : xop { private weak string[] sref; public xoplist (int x, string[] s) { myindex = x; sref = s; } public override void vrun () { foreach (string l in sref) { outputstring = (outputstring + l + "\n"); } } } public class xopedit : xop { public weak string[] sref; private int targidx; private string replacewith; public xopedit (int x, string[] s, int i, string r) { myindex = x; sref = s; targidx = i; replacewith = r; } public override void vrun () { sref[targidx] = replacewith; } wb } public string concats (string[] s) { string o = ""; foreach (string l in s) { o = (o + l); } return o; } public string lists (string[] s) { string o = ""; foreach (string l in s) { o = (o + l + "\n"); } return o; } public void replaces (ref string[] s, int x, string w) { s[x] = w; } xop[] ops; void main () { GLib.Intl.setlocale(ALL,""); int rounds = 100; string[] s = {"one","two","three","four","five"}; print("original string[] is {%s}\n\n",string.joinv(",",s)); ops += new xopconcat(0,s); ops += new xoplist(1,s); ops += new xopedit(2,s,4,"五"); int64[] scsum = new int64[rounds]; for (int i = 0; i < rounds; i++) { int64 tts = GLib.get_monotonic_time(); s[4] = "five"; ops[0].outputstring = ""; ops[1].outputstring = ""; ops[0].vrun(); ops[1].vrun(); ops[2].vrun(); int64 tte = GLib.get_monotonic_time(); scsum[i] = (tte - tts); } //foreach (int64 i in scsum) {print("%.2f ", ((double) i)); } snortintsixtyfour(scsum); double scmed = getmedianintsixtyfour(scsum); print("\nsubclassed objects took a median of %.2f microseconds over %d iterations\n",scmed,rounds); print("subclassed concat: %s\n",ops[0].outputstring); print("subclassed list:\n%s",ops[1].outputstring); print("subclass altered string[] is {%s}\n",string.joinv(",",s)); s = {"one","two","three","four","five"}; int64[] fnsum = new int64[rounds]; string listoutput = ""; string concatoutput = ""; for (int i = 0; i < rounds; i++) { int64 tts = GLib.get_monotonic_time(); s[4] = "five"; listoutput = ""; concatoutput = ""; listoutput = lists(s); concatoutput = concats(s); replaces(ref s,4,"五"); int64 tte = GLib.get_monotonic_time(); fnsum[i] = (tte - tts); } snortintsixtyfour(fnsum); double fnmed = getmedianintsixtyfour(fnsum); print("\nfunction code took a median of %.2f microseconds over %d iterations\n",fnmed,rounds); print("function concat: %s\n",concatoutput); print("function list:\n%s",listoutput); print("function altered string[] is {%s}\n",string.joinv(",",s)); s = {"one","two","three","four","five"}; listoutput = ""; concatoutput = ""; int64[] ilsum = new int64[rounds]; for (int i = 0; i < rounds; i++) { int64 tts = GLib.get_monotonic_time(); s[4] = "five"; listoutput = ""; concatoutput = ""; foreach (string l in s) { concatoutput = (concatoutput + l); } foreach (string l in s) { listoutput = (listoutput + l + "\n"); } s[4] = "五"; int64 tte = GLib.get_monotonic_time(); ilsum[i] = (tte - tts); } snortintsixtyfour(ilsum); double ilmed = getmedianintsixtyfour(ilsum); print("\ninline code took a median of %f microseconds over %d iterations\n",ilmed,rounds); print("inline concat: %s\n",concatoutput); print("inline list:\n%s",listoutput); print("inline altered string[] is {%s}\n",string.joinv(",",s)); print("\ninline is %.2f%% faster than subclassed\n",(100 - ((ilmed/scmed)) * 100.0)); }
#+RESULTS:
original string[] is {one,two,three,four,five} subclassed objects took a median of 1.00 microseconds over 100 iterations subclassed concat: onetwothreefourfive subclassed list: one two three four five subclass altered string[] is {one,two,three,four,五} function code took a median of 1.00 microseconds over 100 iterations function concat: onetwothreefourfive function list: one two three four five function altered string[] is {one,two,three,four,五} inline code took a median of 1.000000 microseconds over 100 iterations inline concat: onetwothreefourfive inline list: one two three four five inline altered string[] is {one,two,three,four,五} inline is 0.00% faster than subclassed
PACKETS:vala:

Get lists of ints in range by nth + offset, using max list size

Useful for threading x items over y threads.
Nth & offset useful for skipping items, like quoted parts of strings for example:

// make y packets from a 0-x range by nth z, with an offset of u // by c.p.brown 2024 // used for multithreading struct packet { public int[] items; } packet[] packseq (int ind, int[] ix, int c, bool q, int y, int z, int u, string m) { // x = items to package // q = use items if true, use items count if false, eg {2,12,34,43} vs {0,1,2,3} // y = max packets, usually thread-count // z = nth item to package, eg {0,1,2,3,4,5} nth 2 is {0,2,4} // u = offset items, eg {0,1,2,3,4,5} offset 2 is {2,3,4,5}, {0,1,2,3,4,5} nth 2 offset 1 is {1,3,5} // m = mode "" is uniform packets, "BIAS" is small to large packets packet[] o = new packet[0]; int[] srcitems = ix; int maxpackets = y; int nthsrc = z; // nth int srcoffset = u; // offset int srclen = c; int[] sampleditems = {}; int[] itempacket = {}; if (q) { srclen = srcitems.length; for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += srcitems[j]; } } else { for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += j; } } if (sampleditems.length > maxpackets) { if (m == "BIAS") { int qb = 0; int dl = sampleditems.length; for (int n = 0; n < dl; n++) { double x = (n * 1.0) / (dl - 1.0); double s = 0.8; double g = 0.0; double f = 1.0; if (x < 1.5) { g = ((x * f) / ((((1.0 / s) - 2.0) * (1.0 - (f * x))) + 1.0)) / f; } else { g = ((((x * 2.0) - 1.0) / ((((1.0 / (1.0 - s)) - 2.0) * (1.0 - ( (2.0 * x) - 1.0))) + 1.0)) / 2.0) + 0.5; } double w = maxpackets - 1; double h = Math.ceil(g * w); int bar = ((int) h); if (bar > (qb + 1)) { bar = qb + 1; } itempacket += bar; qb = bar; } o = new packet[maxpackets]; for (int i = 0; i < maxpackets; i++) { o[i] = new packet(); } for (int r = 0; r < sampleditems.length; r++) { for (int i = 0; i < maxpackets; i++) { if (itempacket[r] == i) { o[i].items += sampleditems[r]; } } } // correct anomolous packet item counts... this should be prevented @ sorurce for (int i = 1; i < o.length; i++) { if (o[i-1].items.length > o[i].items.length) { o[i].items += o[i-1].items[(o[i-1].items.length - 1)]; o[i-1].items.resize(o[i-1].items.length - 1); } } return o; } else { int itemsperpacket = (sampleditems.length / maxpackets); double neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); int mxn = 0; while (neededpackets > maxpackets) { if (mxn > 1000000) { return o; } itemsperpacket += 1; neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); mxn += 1; } int np = (int) neededpackets; int dregs = sampleditems.length - (itemsperpacket * ((int) neededpackets)); int h = 0; if (dregs > 0) { h = 1; } if ((np + h) > 0) { o = new packet[(np+h)]; int f = 0; for (int i = 0; i < np; i++) { o[i] = new packet(); for (int t = 0; t < itemsperpacket; t++) { o[i].items += sampleditems[f]; f += 1; } } if (dregs > 0) { o[np] = new packet(); for (int t = 0; t < dregs; t++) { o[np].items += sampleditems[f]; f += 1; } } } } } else { // one item per packet for (int k = 0; k < sampleditems.length; k++) { packet oo = new packet(); oo.items += sampleditems[k]; o += oo; } } return o; } string printintarray (int[] s, int f) { string[] ii = {"", "\"", "{", "[", "[", "| ", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%d".printf(s[i]) + dd[f]); } o = (o + "%d".printf(s[s.length - 1]) + oo[f]); return o; } void main() { GLib.Intl.setlocale(ALL,""); int[] nup = new int[0]; packet[] p = packseq(0,nup,110,false,16,1,0,"BIAS"); for (int x = 0; x < p.length; x++) { print("[%03d/%03d] %s\n",x,(p.length - 1),printintarray(p[x].items,2)); } print("\n"); p = packseq(0,nup,33,false,16,1,0,""); for (int x = 0; x < p.length; x++) { print("%s\n",printintarray(p[x].items,2)); } print("\n"); p = packseq(0,nup,100,false,16,2,1,""); for (int x = 0; x < p.length; x++) { print("%s\n",printintarray(p[x].items,2)); } print("\n"); p = packseq(0,nup,8,false,16,1,0,""); for (int x = 0; x < p.length; x++) { print("%s\n",printintarray(p[x].items,2)); } print("\n"); print("\nweighted packet distribution:\n"); int qb = 1; int dl = 120; for (int n = 0; n < dl; n++) { double x = (n * 1.0) / (dl - 1.0); double s = 0.9; double g = 0.0; double f = 1.0; if (x < 1.5) { g = ((x * f) / ((((1.0 / s) - 2.0) * (1.0 - (f * x))) + 1.0)) / f; } else { g = ((((x * 2.0) - 1.0) / ((((1.0 / (1.0 - s)) - 2.0) * (1.0 - ( (2.0 * x) - 1.0))) + 1.0)) / 2.0) + 0.5; } double w = 15.0; double h = Math.ceil(g * w); int bar = ((int) h); string plot = ""; if (bar > (qb + 1)) { bar = qb + 1; } for (int j = 0; j < bar; j++) { plot = plot + "█"; } qb = bar; print("%02d | %0.2f | %0.4f\t| %0.2f\t| %02d | %s\n",n,x,g,h,bar,plot); } }
#+RESULTS:
Compilation succeeded - 4 warning(s) [000/015] {0} [001/015] {1} [002/015] {2,3} [003/015] {5,6,4} [004/015] {7,8,9} [005/015] {10,11,12} [006/015] {13,14,15} [007/015] {16,17,18,19} [008/015] {20,21,22,23,24} [009/015] {25,26,27,28,29} [010/015] {30,31,32,33,34,35,36} [011/015] {37,38,39,40,41,42,43,44} [012/015] {45,46,47,48,49,50,51,52,53,54} [013/015] {55,56,57,58,59,60,61,62,63,64,65,66,67} [014/015] {68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84} [015/015] {85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109} {0,1,2} {3,4,5} {6,7,8} {9,10,11} {12,13,14} {15,16,17} {18,19,20} {21,22,23} {24,25,26} {27,28,29} {30,31,32} {1,3,5,7} {9,11,13,15} {17,19,21,23} {25,27,29,31} {33,35,37,39} {41,43,45,47} {49,51,53,55} {57,59,61,63} {65,67,69,71} {73,75,77,79} {81,83,85,87} {89,91,93,95} {97,99} {0} {1} {2} {3} {4} {5} {6} {7} weighted packet distribution: 00 | 0.00 | 0.0000 | 0.00 | 00 | 01 | 0.01 | 0.0709 | 2.00 | 01 | █ 02 | 0.02 | 0.1333 | 2.00 | 02 | ██ 03 | 0.03 | 0.1888 | 3.00 | 03 | ███ 04 | 0.03 | 0.2384 | 4.00 | 04 | ████ 05 | 0.04 | 0.2830 | 5.00 | 05 | █████ 06 | 0.05 | 0.3234 | 5.00 | 05 | █████ 07 | 0.06 | 0.3600 | 6.00 | 06 | ██████ 08 | 0.07 | 0.3934 | 6.00 | 06 | ██████ 09 | 0.08 | 0.4241 | 7.00 | 07 | ███████ 10 | 0.08 | 0.4523 | 7.00 | 07 | ███████ 11 | 0.09 | 0.4783 | 8.00 | 08 | ████████ 12 | 0.10 | 0.5023 | 8.00 | 08 | ████████ 13 | 0.11 | 0.5247 | 8.00 | 08 | ████████ 14 | 0.12 | 0.5455 | 9.00 | 09 | █████████ 15 | 0.13 | 0.5649 | 9.00 | 09 | █████████ 16 | 0.13 | 0.5830 | 9.00 | 09 | █████████ 17 | 0.14 | 0.6000 | 10.00 | 10 | ██████████ 18 | 0.15 | 0.6160 | 10.00 | 10 | ██████████ 19 | 0.16 | 0.6310 | 10.00 | 10 | ██████████ 20 | 0.17 | 0.6452 | 10.00 | 10 | ██████████ 21 | 0.18 | 0.6585 | 10.00 | 10 | ██████████ 22 | 0.18 | 0.6712 | 11.00 | 11 | ███████████ 23 | 0.19 | 0.6832 | 11.00 | 11 | ███████████ 24 | 0.20 | 0.6945 | 11.00 | 11 | ███████████ 25 | 0.21 | 0.7053 | 11.00 | 11 | ███████████ 26 | 0.22 | 0.7156 | 11.00 | 11 | ███████████ 27 | 0.23 | 0.7254 | 11.00 | 11 | ███████████ 28 | 0.24 | 0.7347 | 12.00 | 12 | ████████████ 29 | 0.24 | 0.7436 | 12.00 | 12 | ████████████ 30 | 0.25 | 0.7521 | 12.00 | 12 | ████████████ 31 | 0.26 | 0.7602 | 12.00 | 12 | ████████████ 32 | 0.27 | 0.7680 | 12.00 | 12 | ████████████ 33 | 0.28 | 0.7755 | 12.00 | 12 | ████████████ 34 | 0.29 | 0.7826 | 12.00 | 12 | ████████████ 35 | 0.29 | 0.7895 | 12.00 | 12 | ████████████ 36 | 0.30 | 0.7961 | 12.00 | 12 | ████████████ 37 | 0.31 | 0.8024 | 13.00 | 13 | █████████████ 38 | 0.32 | 0.8085 | 13.00 | 13 | █████████████ 39 | 0.33 | 0.8144 | 13.00 | 13 | █████████████ 40 | 0.34 | 0.8200 | 13.00 | 13 | █████████████ 41 | 0.34 | 0.8255 | 13.00 | 13 | █████████████ 42 | 0.35 | 0.8308 | 13.00 | 13 | █████████████ 43 | 0.36 | 0.8359 | 13.00 | 13 | █████████████ 44 | 0.37 | 0.8408 | 13.00 | 13 | █████████████ 45 | 0.38 | 0.8455 | 13.00 | 13 | █████████████ 46 | 0.39 | 0.8501 | 13.00 | 13 | █████████████ 47 | 0.39 | 0.8545 | 13.00 | 13 | █████████████ 48 | 0.40 | 0.8588 | 13.00 | 13 | █████████████ 49 | 0.41 | 0.8630 | 13.00 | 13 | █████████████ 50 | 0.42 | 0.8671 | 14.00 | 14 | ██████████████ 51 | 0.43 | 0.8710 | 14.00 | 14 | ██████████████ 52 | 0.44 | 0.8748 | 14.00 | 14 | ██████████████ 53 | 0.45 | 0.8785 | 14.00 | 14 | ██████████████ 54 | 0.45 | 0.8820 | 14.00 | 14 | ██████████████ 55 | 0.46 | 0.8855 | 14.00 | 14 | ██████████████ 56 | 0.47 | 0.8889 | 14.00 | 14 | ██████████████ 57 | 0.48 | 0.8922 | 14.00 | 14 | ██████████████ 58 | 0.49 | 0.8954 | 14.00 | 14 | ██████████████ 59 | 0.50 | 0.8985 | 14.00 | 14 | ██████████████ 60 | 0.50 | 0.9015 | 14.00 | 14 | ██████████████ 61 | 0.51 | 0.9044 | 14.00 | 14 | ██████████████ 62 | 0.52 | 0.9073 | 14.00 | 14 | ██████████████ 63 | 0.53 | 0.9101 | 14.00 | 14 | ██████████████ 64 | 0.54 | 0.9128 | 14.00 | 14 | ██████████████ 65 | 0.55 | 0.9155 | 14.00 | 14 | ██████████████ 66 | 0.55 | 0.9181 | 14.00 | 14 | ██████████████ 67 | 0.56 | 0.9206 | 14.00 | 14 | ██████████████ 68 | 0.57 | 0.9231 | 14.00 | 14 | ██████████████ 69 | 0.58 | 0.9255 | 14.00 | 14 | ██████████████ 70 | 0.59 | 0.9278 | 14.00 | 14 | ██████████████ 71 | 0.60 | 0.9301 | 14.00 | 14 | ██████████████ 72 | 0.61 | 0.9324 | 14.00 | 14 | ██████████████ 73 | 0.61 | 0.9346 | 15.00 | 15 | ███████████████ 74 | 0.62 | 0.9367 | 15.00 | 15 | ███████████████ 75 | 0.63 | 0.9388 | 15.00 | 15 | ███████████████ 76 | 0.64 | 0.9409 | 15.00 | 15 | ███████████████ 77 | 0.65 | 0.9429 | 15.00 | 15 | ███████████████ 78 | 0.66 | 0.9448 | 15.00 | 15 | ███████████████ 79 | 0.66 | 0.9467 | 15.00 | 15 | ███████████████ 80 | 0.67 | 0.9486 | 15.00 | 15 | ███████████████ 81 | 0.68 | 0.9505 | 15.00 | 15 | ███████████████ 82 | 0.69 | 0.9523 | 15.00 | 15 | ███████████████ 83 | 0.70 | 0.9540 | 15.00 | 15 | ███████████████ 84 | 0.71 | 0.9558 | 15.00 | 15 | ███████████████ 85 | 0.71 | 0.9574 | 15.00 | 15 | ███████████████ 86 | 0.72 | 0.9591 | 15.00 | 15 | ███████████████ 87 | 0.73 | 0.9607 | 15.00 | 15 | ███████████████ 88 | 0.74 | 0.9623 | 15.00 | 15 | ███████████████ 89 | 0.75 | 0.9639 | 15.00 | 15 | ███████████████ 90 | 0.76 | 0.9654 | 15.00 | 15 | ███████████████ 91 | 0.76 | 0.9669 | 15.00 | 15 | ███████████████ 92 | 0.77 | 0.9684 | 15.00 | 15 | ███████████████ 93 | 0.78 | 0.9699 | 15.00 | 15 | ███████████████ 94 | 0.79 | 0.9713 | 15.00 | 15 | ███████████████ 95 | 0.80 | 0.9727 | 15.00 | 15 | ███████████████ 96 | 0.81 | 0.9741 | 15.00 | 15 | ███████████████ 97 | 0.82 | 0.9754 | 15.00 | 15 | ███████████████ 98 | 0.82 | 0.9767 | 15.00 | 15 | ███████████████ 99 | 0.83 | 0.9780 | 15.00 | 15 | ███████████████ 100 | 0.84 | 0.9793 | 15.00 | 15 | ███████████████ 101 | 0.85 | 0.9806 | 15.00 | 15 | ███████████████ 102 | 0.86 | 0.9818 | 15.00 | 15 | ███████████████ 103 | 0.87 | 0.9830 | 15.00 | 15 | ███████████████ 104 | 0.87 | 0.9842 | 15.00 | 15 | ███████████████ 105 | 0.88 | 0.9854 | 15.00 | 15 | ███████████████ 106 | 0.89 | 0.9866 | 15.00 | 15 | ███████████████ 107 | 0.90 | 0.9877 | 15.00 | 15 | ███████████████ 108 | 0.91 | 0.9888 | 15.00 | 15 | ███████████████ 109 | 0.92 | 0.9899 | 15.00 | 15 | ███████████████ 110 | 0.92 | 0.9910 | 15.00 | 15 | ███████████████ 111 | 0.93 | 0.9921 | 15.00 | 15 | ███████████████ 112 | 0.94 | 0.9931 | 15.00 | 15 | ███████████████ 113 | 0.95 | 0.9941 | 15.00 | 15 | ███████████████ 114 | 0.96 | 0.9952 | 15.00 | 15 | ███████████████ 115 | 0.97 | 0.9962 | 15.00 | 15 | ███████████████ 116 | 0.97 | 0.9971 | 15.00 | 15 | ███████████████ 117 | 0.98 | 0.9981 | 15.00 | 15 | ███████████████ 118 | 0.99 | 0.9991 | 15.00 | 15 | ███████████████ 119 | 1.00 | 1.0000 | 15.00 | 15 | ███████████████
PARSEXML:vala:

xml parsing.

Obsolete test:
This has since been replaced by more comprehensive multithreaded version, which is too large to put here.

// xml parsing workout // by c.p.brown 2024 // doesn't support namespaces or lazy closures "/>" struct packet { int[] items; } packet[] packseq (int x, int y, int z, int u) { print("packseq (\n\t%d,\n\t%d,\n\t%d,\n\t%d\n)\n",x,y,z,u); // 0 1 2 3 4 5 6 7 8 9 by 2 offset 1 = // 1 3 5 7 8 packet[] o = new packet[0]; int n = x; int p = y; int q = z; // nth int m = u; // offset int[] nn = {}; for (int j = m; j < n; j += q) { nn += j; } int s = nn.length / p; int r = nn.length - (p * s); print("seq %d by %d offset %d (%d) contains %d packets of %d, plus %d leftovers = %d\n",x,q,m,nn.length,s,p,r,((p * s)+r)); int h = 0; if (r > 0) { h = 1; } if ((s + h) > 0) { o = new packet[(s+h)]; int f = 0; for (int i = 0; i < s; i++) { o[i] = new packet(); for (int t = 0; t < p; t++) { o[i].items += nn[f]; f += 1; } } if (r > 0) { o[s] = new packet(); for (int t = 0; t < r; t++) { o[s].items += nn[f]; f += 1; } } } return o; } public class delimitquotes { private int[] chunks; private weak string[] splits; private string quoteopen; private string quoteclose; private string quotedelimiter; private int quoteopenlength; public delimitquotes (string q, string d, int[] c, string[] s) { splits = s; chunks = c; quoteopen = q.get_char(q.index_of_nth_char(0)).to_string(); quoteclose = q.get_char(q.index_of_nth_char(1)).to_string(); quoteopenlength = quoteopen.length; quotedelimiter = d; } public void run() { foreach (int x in chunks) { StringBuilder b = new StringBuilder(splits[x]); print("delimitquote processing string[%d] %s\n",x,splits[x].replace("\n","")); int ii = 0; int nxt = 0; int n = 0; while ((ii != -1) && (n < 1000000)) { ii = b.str.index_of(quoteopen,nxt); if (ii != -1) { nxt = ii; b.erase(ii,quoteopenlength); b.insert(ii,quotedelimiter); } n += 1; } b.append(quotedelimiter); print("delimitquote returned string[%d] %s\n",x,b.str.replace("\n","")); splits[x] = b.str; } } } public class delimitescapedquotes { private int[] chunks; private weak string[] splits; private string quoteopen; private string quoteclose; private string escapedcharstring; private string escapedopenquote; private string escapedclosequote; private string escapedquotedelimiter; private int escapedclosequotelength; private int escapedopenquotelength; public delimitescapedquotes (string q, string e, string d, int[] c, string[] s) { splits = s; chunks = c; quoteopen = q.get_char(q.index_of_nth_char(0)).to_string(); quoteclose = q.get_char(q.index_of_nth_char(1)).to_string(); escapedcharstring = e; escapedopenquote = (escapedcharstring + quoteopen); escapedopenquotelength = escapedopenquote.length; escapedclosequote = (escapedcharstring + quoteclose); escapedclosequotelength = escapedclosequote.length; escapedquotedelimiter = d; } public void run() { foreach (int x in chunks) { StringBuilder b = new StringBuilder(splits[x]); print("delimitescapedquote processing string[%d] %s\n",x,splits[x].replace("\n","")); int ii = 0; int nxt = 0; int n = 0; while ((ii != -1) && (n < 1000000)) { ii = b.str.index_of(escapedopenquote,nxt); if (ii != -1) { nxt = ii; b.erase(ii,escapedopenquotelength); b.insert(ii,escapedquotedelimiter); } n += 1; } b.append(escapedquotedelimiter); print("delimitescapedquote returned string[%d] %s\n",x,b.str.replace("\n","")); splits[x] = b.str; } } } public class restoreescapedquotes { private int[] chunks; private weak string[] splits; private string quoteopen; private string quoteclose; private string escapedcharstring; private string escapedopenquote; private string escapedclosequote; private int escapedclosequotelength; private int escapedopenquotelength; public restoreescapedquotes (string q, string e, int[] c, string[] s) { splits = s; chunks = c; quoteopen = q.get_char(q.index_of_nth_char(0)).to_string(); quoteclose = q.get_char(q.index_of_nth_char(1)).to_string(); escapedcharstring = e; escapedopenquote = (escapedcharstring + quoteopen); escapedopenquotelength = escapedopenquote.length; escapedclosequote = (escapedcharstring + quoteclose); escapedclosequotelength = escapedclosequote.length; } public void run() { foreach (int x in chunks) { splits[x] = escapedopenquote + splits[x] + escapedclosequote; } } } public class uqelimiters { private int[] chunks; private weak string[] unquoted; private string rowenddelimiter; private string colenddelimiter; private string rowstartdelimiter; private string colstartdelimiter; private string rowend; private string colend; private string rowstartopen; private string rowstartclose; private string colstartopen; private string colstartclose; private string quoteopen; private string quoteclose; private bool docolwildcard; private bool dorowwildcard; private bool docols; private bool dorows; private bool doquotes; private bool dosamequote; public uqelimiters ( string[] rr, int[] chu, bool dco, bool cwc, string cso, string csc, string cte, string cds, string cde, bool dro, bool rwc, string rso, string rsc, string rte, string rds, string rde, string quo, string quc, bool doq, bool dsq ) { chunks = chu; unquoted = rr; docols = dco; docolwildcard = cwc; colstartopen = cso; colstartclose= csc; colend = cte; colstartdelimiter = cds; colenddelimiter = cde; rowend = rte; dorows = dro; dorowwildcard = rwc; rowstartopen = rso; rowstartclose = rsc; rowstartdelimiter = rds; rowenddelimiter = rde; quoteopen = quo; quoteclose = quc; doquotes = doq; dosamequote = dsq; print("uqelimit(\n\tcolstartopen : %s\n\tcolstartclose : %s\n\tcolend : %s\n\tcolstartdelimiter : %s\n\tcolenddelimiter : %s\n\trowstartopen : %s\n\trowstartclose : %s\n\trowend : %s\n\trowstartdelimiter : %s\n\trowenddelimiter : %s\n)\n", colstartopen, colstartclose, colend, colstartdelimiter, colenddelimiter, rowstartopen, rowstartclose, rowend, rowstartdelimiter, rowenddelimiter ); } public void run () { int rowenddelimitercharcount = rowenddelimiter.char_count(); int rowstartdelimitercharcount = rowstartdelimiter.char_count(); int colenddelimitercharcount = colenddelimiter.char_count(); int colstartdelimitercharcount = colstartdelimiter.char_count(); int colendcharcount = colend.char_count(); int rowendcharcount = rowend.char_count(); int colstartcharcount = colstartopen.char_count(); int rowstartcharcount = rowstartopen.char_count(); foreach (int x in chunks) { print("\tuqelimiters processing string[%d] %s\n",x,unquoted[x].replace("\n","")); if (unquoted[x].char_count() != 0) { StringBuilder b = new StringBuilder(unquoted[x]); // delimit row end if (dorows) { int nxt = 0; int n = 0; int ii = 0; //print("raw b.str is %s\n",b.str.replace("\n","")); while (ii != -1) { if (n > 1000000) { break; } ii = b.str.index_of(rowend,nxt); if (ii != -1) { nxt = ii + rowenddelimitercharcount; b.erase(ii,rowendcharcount); b.insert(ii,rowenddelimiter); //print("edited b.str is %s\n",b.str.replace("\n","")); } else { break; } n += 1; } if (dorowwildcard) { nxt = 0; n = 0; ii = 0; //print("raw b.str is %s\n",b.str.replace("\n","")); while (ii != -1) { if (n > 1000000) { break; } ii = b.str.index_of(rowstartopen,nxt); if (ii != -1) { nxt = ii + rowstartdelimitercharcount; int oo = b.str.index_of(rowstartclose,ii); if (oo != -1) { b.erase(ii,((oo+1) - ii)); b.insert(ii,rowstartdelimiter); } else { // rowstartopen is found, but not rowstartclose, // meaning the tag is probably split by quoted chars b.erase(ii,-1); b.insert(ii,rowstartdelimiter); } } else { break; } n += 1; } } else { nxt = 0; n = 0; ii = 0; while (ii != -1) { if (n > 1000000) { break; } ii = b.str.index_of(rowstartopen,nxt); if (ii != -1) { nxt = ii + rowstartdelimitercharcount; b.erase(ii,rowstartcharcount); b.insert(ii,rowstartdelimiter); } else { break; } n += 1; } } } // delimit col end if (docols) { int nxt = 0; int n = 0; int ii = 0; while (ii != -1) { if (n > 1000000) { break; } ii = b.str.index_of(colend,nxt); if (ii != -1) { nxt = ii + colenddelimitercharcount; b.erase(ii,colendcharcount); b.insert(ii,colenddelimiter); //print("edited b.str is %s\n",b.str.replace("\n","")); } else { break; } n += 1; } if (docolwildcard) { nxt = 0; n = 0; ii = 0; //print("raw b.str is %s\n",b.str.replace("\n","")); while (ii != -1) { if (n > 1000000) { break; } ii = b.str.index_of(colstartopen,nxt); if (ii != -1) { nxt = ii + colstartdelimitercharcount; int oo = b.str.index_of(colstartclose,ii); if (oo != -1) { b.erase(ii,((oo+1) - ii)); b.insert(ii,colstartdelimiter); //print("edited b.str is %s\n",b.str.replace("\n","")); } else { // can't find colstartclose, // assuming its cut by a quoted string... b.erase(ii,-1); b.insert(ii,colstartdelimiter); } } else { break; } n += 1; } } else { nxt = 0; n = 0; ii = 0; while (ii != -1) { if (n > 1000000) { break; } ii = b.str.index_of(colstartdelimiter,nxt); if (ii != -1) { nxt = ii + colstartdelimitercharcount; b.erase(ii,colstartcharcount); b.insert(ii,colstartdelimiter); } else { break; } n += 1; } } } unquoted[x] = b.str; print("\tuqelimiters returned string[%d] %s\n",x,unquoted[x].replace("\n","")); //print("\nreturning -----------------------------------------------------\n%s\n",unquoted[x]); } } //print("unq=\n%s\n",string.joinv("",unquoted)); } } public class restorequotes { private int[] chunks; private weak string[] quoted; private string quoteopen; private string quoteclose; public restorequotes (int[] chu, string[] qst, string quo, string quc) { chunks = chu; quoted = qst; quoteopen = quo; quoteclose = quc; } public void run () { foreach (int x in chunks) { if (quoted[x].char_count() > 0) { quoted[x] = quoteopen + quoted[x] + quoteclose; } else { quoted[x] = "[QO]" + "[QC]"; } } } } public class procrowchunk { private int[] chunks; private weak string[] rows; private string colstartopen; private string colstartclose; private string rowstartopen; private string rowstartclose; private bool dorows; private bool docols; private bool dorowwildcard; private bool docolwildcard; private bool doquotes; private string colstartdelimiter; private string colstartclosedelimiter; private string colenddelimiter; private string rowstartdelimiter; private string rowenddelimiter; private unichar escapechar; private string escapedquotedelimiter; private bool doescapedquotes; private string quotes; public procrowchunk ( string[] rr, int[] chu, string cso, string csc, string rso, string rsc, bool dor, bool doc, bool drw, bool dcw, bool dqo, string csd, string cscd, string ctd, string rsd, string rtd, string quo, unichar ecc, string eqd, bool deq ) { chunks = chu; rows = rr; colstartopen = cso; colstartclose = csc; rowstartopen = rso; rowstartclose = rsc; dorows = dor; docols = doc; dorowwildcard = drw; docolwildcard = dcw; doquotes = dqo; colstartdelimiter = csd; colstartclosedelimiter = cscd; colenddelimiter = ctd; rowstartdelimiter = rsd; rowenddelimiter = rtd; quotes = quo; escapechar = ecc; escapedquotedelimiter = eqd; doescapedquotes = deq; print("procrowchunk(\n\tcolstartopen : %s\n\tcolstartclose : %s\n\tcolstartdelimiter : %s\n\tcolstartclosedelimiter : %s\n\tcolenddelimiter : %s\n\trowstartopen : %s\n\trowstartclose : %s\n\trowstartdelimiter : %s\n\trowenddelimiter : %s\n\tescapedquotedelimiter : %s\n)\n", colstartopen, colstartclose, colstartdelimiter, colstartclosedelimiter, colenddelimiter, rowstartopen, rowstartclose, rowstartdelimiter, rowenddelimiter, escapedquotedelimiter ); } public void run () { int hl = 1000000; int colenddelimitercharcount = colenddelimiter.char_count(); int rowenddelimitercharcount = colenddelimiter.char_count(); int colstartdelimitercharcount = colstartdelimiter.char_count(); int colstartclosedelimitercharcount = colstartclosedelimiter.char_count(); int rowstartdelimitercharcount = rowstartdelimiter.char_count(); int colstartclosecharcount = colstartclose.char_count(); int escapedquotedelimiterlength = escapedquotedelimiter.length; string? quoteopen = null; string? quoteclose = null; unichar? quoteopenchar = null; unichar? quoteclosechar = null; unichar colstartclosechar = colstartclose.get_char(0); string escapecharstring = escapechar.to_string(); if (doquotes) { quoteopen = quotes.get_char(0).to_string(); quoteclose = quotes.get_char(1).to_string(); } foreach (int x in chunks) { if (rows[x].char_count() != 0) { StringBuilder b = new StringBuilder(rows[x]); print("processing row %s\n",b.str.replace("\n","")); // remove everything to rowopen or colopen, whichever is 1st if (dorows && docols) { int nxt = 0; int n = 0; int ii = 0; while (ii != -1) { if (n > hl) { break; } ii = b.str.index_of(colstartdelimiter,nxt); if (ii != -1) { nxt = 0; b.erase(0,ii); } else { break; } n += 1; } } // remove rowstartopen to colstartopen if (dorows && docols) { int nxt = 0; int n = 0; int ii = 0; while (ii != -1) { if (n > hl) { break; } ii = b.str.index_of(rowstartdelimiter,nxt); if (ii != -1) { nxt = ii + rowenddelimitercharcount; int oo = b.str.index_of(colstartdelimiter,ii); if (oo != -1) { b.erase(ii,(oo - ii)); } } else { break; } n += 1; } } // remove colend to colstartopen, or rowend if its the last column if (docols) { int nxt = 0; int n = 0; int ii = 0; while (ii != -1) { if (n > hl) { break; } ii = b.str.index_of(colenddelimiter,nxt); if (ii != -1) { nxt = ii + colenddelimiter.length; int oo = b.str.index_of(colstartdelimiter,ii); if (oo != -1) { print("[%.2d] erasing from %d to %d {%s}\n",n,nxt,oo,b.str.substring(nxt,((oo - ii) - colenddelimiter.length))); b.erase(nxt,(oo - nxt)); print("[%.2d] erased {%s}\n",n,b.str); } } else { break; } n += 1; } // remove colstartopen to colstartclose if (docolwildcard) { if (doquotes) { // look-ahead past quote pairs // TODO: find a cheaper way to do this... nxt = 0; ii = 0; n = 0; while (ii != -1) { if (n > hl) { break; } ii = b.str.index_of(colstartdelimiter,nxt); if (ii != -1) { nxt = ii + colstartdelimiter.length; int ci = -1; int i = 0; unichar c = 0; bool amquoted = false; while (b.str.get_next_char(ref i, out c)) { if (i < ii) { continue; } if (c == escapechar) { b.str.get_next_char(ref i, out c); continue; } if (!amquoted && (c == quoteopenchar)) { amquoted = true; continue; } if (amquoted && (c == quoteclosechar)) { amquoted = false; continue; } if (!amquoted && (c == colstartclosechar)) { ci = i; break; } } if (ci != -1) { print("erasing from %d to %d {%s}\n",nxt,ci,b.str.substring(nxt,(ci - nxt))); b.erase(nxt,(ci - nxt)); } } n += 1; } } } } if (doescapedquotes) { int nxt = 0; int n = 0; int ii = 0; int z = 0; string reqs = ""; while (ii != -1) { if (n > hl) { break; } ii = b.str.index_of(escapedquotedelimiter,nxt); if (ii != -1) { nxt = ii; b.erase(ii,escapedquotedelimiterlength); reqs = escapecharstring + quotes.get_char(quotes.index_of_nth_char(z)).to_string(); print("%d of %s is %s\n",z,quotes,reqs); b.insert(ii,reqs); z = (z ^ 1 ); } else { break; } n += 1; } } rows[x] = b.str; } } } } void main() { GLib.Intl.setlocale(ALL,""); int nthr = (int) GLib.get_num_processors(); StringBuilder sb = new StringBuilder(""" “junk” <r junk> <c>row_0 col_0</c> <c>row_0 \“col_1\”</c> </r> “<r>” junk <r> junk <c t=“s”>“<c t=\“s\”>\“row_1 col_0\”</c>”</c> junk <c t=“s”>row_1 col_1</c>junk </r>junk"""); string colstart = "<c*>"; string rowstart = "<r*>"; string colend = "</c>"; string rowend = "</r>"; string quotes = "“”"; string escaper = "\\"; unichar escapechar = '\\'; char coldelimitchar = '\x1E'; char rowdelimitchar = '\x1F'; char colstartdelimitchar = '\x1C'; char rowstartdelimitchar = '\x1D'; char colstartclosedelimitchar = '\x1A'; char escapedquotedelimiterchar = '\x1B'; char quotedelimiterchar = '\x15'; string cdc = "%c".printf(coldelimitchar); string rdc = "%c".printf(rowdelimitchar); string cso = "%c".printf(colstartdelimitchar); string cscd = "%c".printf(colstartclosedelimitchar); string rso = "%c".printf(rowstartdelimitchar); string qdc = "%c".printf(quotedelimiterchar); string escapedquotedelimiter = "%c".printf(escapedquotedelimiterchar); print("column close delimiter : %s\n",cdc); print("row close delimiter : %s\n",rdc); print("column start open delimiter : %s\n",cso); print("column start close delimiter : %s\n",cscd); print("quote delimiter : %s\n",qdc); print("escaped quote delimiter : %s\n",escapedquotedelimiter); bool dorows = (rowend != null) && (rowend != ""); bool docols = (colend != null) && (colend != ""); bool docolwildcard = false; bool dorowwildcard = false; bool dosamequote = true; bool hasescapedquote = false; bool doescapedquotes = false; bool doquotes = ((quotes != null) && (quotes.char_count() == 2)); string? quoteopen = null; string? quoteclose = null; if (doquotes) { quoteopen = quotes.get_char(quotes.index_of_nth_char(0)).to_string(); quoteclose = quotes.get_char(quotes.index_of_nth_char(1)).to_string(); dosamequote = (strcmp(quoteopen,quoteclose) == 0); } // wildcard open tags string colstartopen = colstart; string rowstartopen = rowstart; string? colstartclose = null; string? rowstartclose = null; int iw = colstart.index_of("*"); if (iw != -1) { colstartopen = colstart.substring(0,iw); colstartclose = colstart.substring(iw+1); docolwildcard = true; } if (docolwildcard) { print("colstartclose is %s\n",colstartclose); } iw = rowstart.index_of("*"); if (iw != -1) { rowstartopen = rowstart.substring(0,iw); rowstartclose = rowstart.substring(iw+1); dorowwildcard = true; } if (dorowwildcard) { print("rowstartclose is %s\n",rowstartclose); } // replace escaped quotes with \x1B if (doquotes) { string escapedquoteclose = escaper + quoteclose; if (sb.str.index_of(escapedquoteclose) != -1) { print("delimiting escaped quotes...\n"); string[] segs = sb.str.split(escapedquoteclose); if (segs.length > 0) { doescapedquotes = true; int ofs = 0; packet[] packets = packseq(segs.length,nthr,1,0); if (packets.length > 1) { ThreadPool<delimitescapedquotes> deq = new ThreadPool<delimitescapedquotes>.with_owned_data((delimitescapedquotes) => { delimitescapedquotes.run(); }, nthr, false ); for (int x = 0; x < packets.length; x++) { deq.add( new delimitescapedquotes ( quotes, escaper, escapedquotedelimiter, packets[x].items, segs ) ); } } else { int[] pak = new int[segs.length]; for (int i = 0; i < pak.length; i++) { pak[i] = i; } delimitescapedquotes deq = new delimitescapedquotes ( quotes, escaper, escapedquotedelimiter, pak, segs ); deq.run(); } } sb.str = string.joinv("",segs); segs.resize(0); } // delimit remaining quotes with \x15 if (sb.str.index_of(quoteclose) != -1) { print("delimiting quotes...\n"); string[] segs = sb.str.split(quoteclose); if (segs.length > 0) { int[] quc = {}; //int[] seglengths = new int[segs.length]; //for (int i = 0; i < segs.length; i++) { seglengths[i] = segs[i].length; } int ofs = 0; packet[] packets = packseq(segs.length,nthr,1,0); if (packets.length > 1) { ThreadPool<delimitquotes> deq = new ThreadPool<delimitquotes>.with_owned_data((delimitquotes) => { delimitquotes.run(); }, nthr, false ); for (int x = 0; x < packets.length; x++) { deq.add( new delimitquotes ( quotes, qdc, packets[x].items, segs ) ); } } else { int[] pak = new int[segs.length]; for (int i = 0; i < pak.length; i++) { pak[i] = i; } delimitquotes deq = new delimitquotes ( quotes, qdc, pak, segs ); deq.run(); } } sb.str = string.joinv("",segs); segs.resize(0); print("\ndelimited quotes:\n%s\n",sb.str); } // split using quotes... print("quote open is %s\n",quoteopen); print("quote close is %s\n",quoteclose); print("quote delimiter is %s\n",qdc); if (sb.str.index_of(qdc) != -1) { string[] quoted = sb.str.split(qdc); sb.str = ""; int ofs = 0; if (strcmp(quoted[0].get_char(0).to_string(), quoteopen) == 0) { ofs = 1; } packet[] packets = packseq(quoted.length,nthr,2,ofs); // in non-quoted strings, replace rowstartopen, colstartopen, rowend, and colend with delimiters... print("number of unquoted segments is %d\n",packets[0].items.length); if (packets.length > 1) { ThreadPool<uqelimiters> qtp = new ThreadPool<uqelimiters>.with_owned_data( (uqelimiters) => { uqelimiters.run(); }, nthr, false ); for (int x = ofs; x < packets.length; x++) { print("creating thread %d for items {",x); for (int i = 0; i < (packets[x].items.length - 1); i++) { print("%d,",packets[x].items[i]); } print("%d}\n",packets[x].items[(packets[x].items.length - 1)]); qtp.add( new uqelimiters( quoted, packets[x].items, docols, docolwildcard, colstartopen, colstartclose, colend, cso, cdc, dorows, dorowwildcard, rowstartopen, rowstartclose, rowend, rso, rdc, quoteopen, quoteclose, doquotes, dosamequote ) ); } } else { int[] pak = {}; for (int i = ofs; i < quoted.length; i += 2) { pak += i; } uqelimiters uqu = new uqelimiters( quoted, pak, docols, docolwildcard, colstartopen, colstartclose, colend, cso, cdc, dorows, dorowwildcard, rowstartopen, rowstartclose, rowend, rso, rdc, quoteopen, quoteclose, doquotes, dosamequote ); uqu.run(); } // restore quotes to quoted strings print("restoring quotes...\n"); ofs = (ofs - 1).abs(); if (packets.length > 1) { ThreadPool<restorequotes> rqo = new ThreadPool<restorequotes>.with_owned_data( (restorequotes) => { restorequotes.run(); }, nthr, false ); for (int x = ofs; x < packets.length; x++) { print("creating thread %d for items {",x); for (int i = 0; i < (packets[x].items.length - 1); i++) { print("%d,",packets[x].items[i]); } print("%d}\n",packets[x].items[(packets[x].items.length - 1)]); rqo.add( new restorequotes( packets[x].items, quoted, quoteopen, quoteclose ) ); } } else { int[] pak = {}; for (int i = ofs; i < quoted.length; i += 2) { pak += i; } print("\t quoteopen is %s\n",quoteopen); print("\t quoteclose is %s\n",quoteclose); restorequotes rqq = new restorequotes( pak, quoted, quoteopen, quoteclose ); rqq.run(); } // rebuild the string... sb.str = string.joinv("",quoted); print("\nrestored quotes:\n%s\n",sb.str); quoted.resize(0); } else { doquotes = false; } } // if there's no quotes, break into rows or columns then delimit... if (!doquotes) { bool allgood = true; int umem = ((int) (sizeof(unichar) * sb.str.char_count())); print("text size is %d bytes\n",umem); string[] segs = {}; // split by rowend delimiter if rowend != null if (dorows) { segs = sb.str.split(rowend); } else { // split by colend if using one row if (docols) { segs = sb.str.split(colend); } else { // its one-cell... check size of string, copy it to dat if its < 1mb, otherwise SOL if (umem < 1000000.1) { segs += sb.str; } else { allgood = false; } } } sb.str = ""; if (allgood) { packet[] packets = packseq(segs.length,nthr,1,0); if (packets.length > 1) { ThreadPool<uqelimiters> qtp = new ThreadPool<uqelimiters>.with_owned_data( (uqelimiters) => { uqelimiters.run(); }, nthr, false ); for (int x = 0; x < packets.length; x++) { qtp.add( new uqelimiters( segs, packets[x].items, docols, docolwildcard, colstartopen, colstartclose, colend, cso, cdc, dorows, dorowwildcard, rowstartopen, rowstartclose, rowend, rso, rdc, quoteopen, quoteclose, doquotes, dosamequote ) ); } } else { int[] pak = new int[segs.length]; for (int i = 0; i < pak.length; i++) { pak[i] = i; } uqelimiters dlm = new uqelimiters( segs, pak, docols, docolwildcard, colstartopen, colstartclose, colend, cso, cdc, dorows, dorowwildcard, rowstartopen, rowstartclose, rowend, rso, rdc, quoteopen, quoteclose, doquotes, dosamequote ); dlm.run(); } if (dorows) { sb.str = string.joinv(rdc,segs); } else { sb.str = string.joinv(cdc,segs); } segs.resize(0); } } // remove last delimiters to prevent extra blank rows/columns if (docols) { int ii = -1; ii = sb.str.last_index_of_char(coldelimitchar); print("last %s is at position %d of %d\n",cdc,ii,sb.str.length); if (ii >= 0) { sb.erase(ii,-1); // erase() doesn't work here, same with truncate() //sb.str = sb.str.substring(0,ii); //print("\nstring with delimiters: %s\n\n",sb.str); } } // if not processing rows, append a row-end delimiter to make the data 1 row of n columns string[] rows = {}; if (!dorows) { sb.append(rdc); dorows = true; } // break into rows, clean up data if (dorows) { rows = sb.str.split(rdc); packet[] packets = packseq(rows.length,nthr,1,0); print("row packet size is %d\n",packets.length); if (packets.length > 1) { // replace unquoted delimiters ThreadPool<procrowchunk> qtp = new ThreadPool<procrowchunk>.with_owned_data( (procrowchunk) => { procrowchunk.run(); }, nthr, false ); for (int x = 0; x < packets.length; x++) { qtp.add( new procrowchunk( rows, packets[x].items, colstartopen, colstartclose, rowstartopen, rowstartclose, dorows, docols, dorowwildcard, docolwildcard, doquotes, cso, cscd, cdc, rso, rdc, quotes, escapechar, escapedquotedelimiter, doescapedquotes ) ); } } else { int[] pak = new int[rows.length]; for (int i = 0; i < pak.length; i++) { pak[i] = i; } procrowchunk urc = new procrowchunk ( rows, pak, colstartopen, colstartclose, rowstartopen, rowstartclose, dorows, docols, dorowwildcard, docolwildcard, doquotes, cso, cscd, cdc, rso, rdc, quotes, escapechar, escapedquotedelimiter, doescapedquotes ); urc.run(); } print("processed rows:\n"); foreach (string r in rows) { print("\t%s\n",r); } } // file data into a 1d array + column count }
#+RESULTS:
Compilation succeeded - 23 warning(s) column close delimiter :  row close delimiter :  column start open delimiter :  column start close delimiter :  quote delimiter :  escaped quote delimiter :  colstartclose is > rowstartclose is > delimiting escaped quotes... packseq ( 4, 16, 1, 0 ) seq 4 by 1 offset 0 (4) contains 0 packets of 16, plus 4 leftovers = 4 delimitescapedquote processing string[0] “junk”<r junk> <c>row_0 col_0</c> <c>row_0 \“col_1 delimitescapedquote returned string[0] “junk”<r junk> <c>row_0 col_0</c> <c>row_0 col_1 delimitescapedquote processing string[1] </c></r>“<r>”junk<r> junk <c t=“s”>“<c t=\“s delimitescapedquote returned string[1] </c></r>“<r>”junk<r> junk <c t=“s”>“<c t=s delimitescapedquote processing string[2] >\“row_1 col_0 delimitescapedquote returned string[2] >row_1 col_0 delimitescapedquote processing string[3] </c>”</c> junk <c t=“s”>row_1 col_1</c>junk</r>junk delimitescapedquote returned string[3] </c>”</c> junk <c t=“s”>row_1 col_1</c>junk</r>junk delimiting quotes... packseq ( 6, 16, 1, 0 ) seq 6 by 1 offset 0 (6) contains 0 packets of 16, plus 6 leftovers = 6 delimitquote processing string[0] “junk delimitquote returned string[0] junk delimitquote processing string[1] <r junk> <c>row_0 col_0</c> <c>row_0 col_1</c></r>“<r> delimitquote returned string[1] <r junk> <c>row_0 col_0</c> <c>row_0 col_1</c></r><r> delimitquote processing string[2] junk<r> junk <c t=“s delimitquote returned string[2] junk<r> junk <c t=s delimitquote processing string[3] >“<c t=s>row_1 col_0</c> delimitquote returned string[3] ><c t=s>row_1 col_0</c> delimitquote processing string[4] </c> junk <c t=“s delimitquote returned string[4] </c> junk <c t=s delimitquote processing string[5] >row_1 col_1</c>junk</r>junk delimitquote returned string[5] >row_1 col_1</c>junk</r>junk delimited quotes: junk <r junk> <c>row_0 col_0</c> <c>row_0 col_1</c> </r> <r> junk <r> junk <c t=s><c t=s>row_1 col_0</c></c> junk <c t=s>row_1 col_1</c>junk </r>junk quote open is “ quote close is ” quote delimiter is  packseq ( 12, 16, 2, 0 ) seq 12 by 2 offset 0 (6) contains 0 packets of 16, plus 6 leftovers = 6 number of unquoted segments is 6 uqelimit( colstartopen : <c colstartclose : > colend : </c> colstartdelimiter :  colenddelimiter :  rowstartopen : <r rowstartclose : > rowend : </r> rowstartdelimiter :  rowenddelimiter :  ) uqelimiters processing string[0] uqelimiters returned string[0] uqelimiters processing string[2] <r junk> <c>row_0 col_0</c> <c>row_0 col_1</c></r> uqelimiters returned string[2]  row_0 col_0 row_0 col_1 uqelimiters processing string[4] junk<r> junk <c t= uqelimiters returned string[4] junk junk  uqelimiters processing string[6] > uqelimiters returned string[6] > uqelimiters processing string[8] </c> junk <c t= uqelimiters returned string[8]  junk  uqelimiters processing string[10] >row_1 col_1</c>junk</r>junk uqelimiters returned string[10] >row_1 col_1junkjunk restoring quotes... quoteopen is “ quoteclose is ” restored quotes: “junk”  row_0 col_0 row_0 col_1  “<r>” junk  junk “s”>“<c t=s>row_1 col_0</c>” junk “s”>row_1 col_1junk junkO][QC] last  is at position 142 of 162 packseq ( 2, 16, 1, 0 ) seq 2 by 1 offset 0 (2) contains 0 packets of 16, plus 2 leftovers = 2 row packet size is 1 procrowchunk( colstartopen : <c colstartclose : > colstartdelimiter :  colstartclosedelimiter :  colenddelimiter :  rowstartopen : <r rowstartclose : > rowstartdelimiter :  rowenddelimiter :  escapedquotedelimiter :  ) processing row “junk” row_0 col_0 row_0 col_1 [00] erasing from 13 to 15 { } [00] erased {row_0 col_0row_0 col_1 } 0 of “” is \“ 1 of “” is \” processing row “<r>”junk junk “s”>“<c t=s>row_1 col_0</c>” junk “s”>row_1 col_1 [00] erasing from 42 to 50 { junk } [00] erased {“s”>“<c t=s>row_1 col_0</c>”“s”>row_1 col_1} erasing from 1 to 9 {“s”>} erasing from 35 to 43 {“s”>} 0 of “” is \“ 1 of “” is \” 0 of “” is \“ 1 of “” is \” processed rows: row_0 col_0row_0 \“col_1\” “<c t=\“s\”>\“row_1 col_0\”</c>”row_1 col_1
PRINT2DARRAY:vala:
// print a 2d array // by c.p.brown 2024 string[,] foldarray (int w, string[] s) { int h = s.length / w; string[,] o = new string[h,w]; for (int i = 0; i < s.length; i++) { int r = i / w; int c = i % w; o[r,c] = s[i]; } return o; } string print2dstringarray (string[,] s) { int w = s.length[1] + 1; int h = s.length[0] + 1; int wcc = "%d".printf(s.length[1]).char_count(); int hcc = "%d".printf(s.length[0]).char_count(); string hline = "---------------------------------------------------------------------------------"; int[] cw = new int [w]; string o = ""; for (int r = 0; r < h; r ++) { if (r == 0) { for (int c = 0; c < w; c ++) { if (c > 0) { cw[c] = wcc; } else { cw[c] = hcc; } } } else { for (int c = 0; c < w; c ++) { if (c == 0) { cw[c] = hcc; } else { //print("row %d col %d s[%d,%d] %s char_count is %d\n",r,c,(r-1),(c-1),s[(r-1),(c-1)],s[(r-1),(c-1)].char_count()); cw[c] = int.max(s[(r-1),(c-1)].char_count(),cw[c]); } } } } for (int r = 0; r < h; r ++) { string ro = ""; if (r == 0) { for (int c = 0; c < w; c++) { if (c == 0) { ro = "%s %.*s |".printf(ro,hcc," "); } else { if (c == (w-1)) { ro = "%s %.*d\n".printf(ro,cw[c],c); } else { ro = "%s %.*d ".printf(ro,cw[c],c); } } } ro = "%s-%.*s-+".printf(ro,hcc,hline); for (int c = 1; c < w; c++) { if (c == (w-1)) { ro = "%s-%.*s-\n".printf(ro,cw[c],hline); } else { ro = "%s-%.*s-".printf(ro,cw[c],hline); } } } else { for (int c = 0; c < w; c ++) { if (c == 0) { ro = "%s %.*d |".printf(ro,hcc,r); } else { int cc = s[(r-1),(c-1)].char_count(); if (c == (w-1)) { ro = "%s %s\n".printf(ro,s[(r-1),(c-1)]); } else { ro = "%s %-*s ".printf(ro,cw[c],s[(r-1),(c-1)]); } } } } o = o + ro; } return o; } void main() { string[] d = {"0","100","2000","30000","A","B","C","DEE","X","Y","ZED","DONE"}; string[,] s = foldarray(4,d); print(print2dstringarray(s)); }
#+RESULTS:
Compilation succeeded - 1 warning(s) | 1 002 0003 00004 | ---+--------------------- 1 | 0 100 2000 30000 2 | A B C DEE 3 | X Y ZED DONE
PRINTF:vala:
void main (string[] args) { string s = "******************"; string t = "text"; int mxd = 12; print("%.*s%s\n",(mxd - t.char_count()),s,t); mxd = 0; print("dotstar 0 +string: %.*s%s\n",int.max(0,(mxd - t.char_count())),s,t); print("star 0 +string: %*s%s\n",int.max(0,(mxd - t.char_count())),s,t); print("%s\n","star 0 nostring : \"%*s\"".printf(0,"").replace(" ","\t")); mxd = 12; print("%*s\n",12,t); print("%*s\n",0,t); print("\"%-*s\"\n",12,t); print("%0*d\n",mxd,1234); // width from elsewhere print("%012d\n",1234); // hardcoded width string r = ("%*s%s").printf(4," ","indent").replace(" ","\t"); print("%s\n",r); print("s contains * : %s\n",((s.index_of("*") != -1)?"true":"false")); GLib.Rand q = new GLib.Rand(); int rch = q.int_range(65,90); print("random char index is %d",rch); print(" (%s)\n",((unichar) rch).to_string()); int[] mch = {65,66,67,68,69,70}; string smch = (string) mch; char* ech; bool siv = (smch.validate(-1,out ech)); print("int array as string is \"%s\"\n",smch); if (siv) { print("string is valid\n"); } StringBuilder sbch = new StringBuilder(""); for (int i = 0; i < mch.length; i++) { sbch.append_unichar(mch[i]); } print("int array as stringbuilder is \"%s\"\n", sbch.str); string lon = "-33.689990"; string lat = "150.546212"; string locurl = "https://www.google.com/search?q=%s+%s".printf(lon,lat); print("locurl = %s\n",locurl); }
#+RESULTS:
********text dotstar 0 +string: text star 0 +string: ******************text star 0 nostring : " " text text "text " 000000001234 000000001234 indent s contains * : true random char index is 71 (G) int array as string is "A" string is valid int array as stringbuilder is "ABCDEF" locurl = https://www.google.com/search?q=-33.689990+150.546212
RECURSIVERADIX:wip::vala:

Test Radix sort on ascii strings.
Fast for a large number of short strings of up to 3 chars, otherwise too slow due to excessive recursion.

DATA ASCII[] P+1 ASCII[] +---+------+ +----+------+ +------+ +----+---+ | 0 | [B]B | | 65 | 2 | | B[B] | | 65 | 1 | | 1 | [B]A |---| 66 | 0, 1 |---| B[A] |---| 66 | 0 | | 2 | [A]B | +----+------+ +------+ +----+---+ +---+------+ || v v +----+------+ | | | 65 | 2 | | | | 66 | 1, 0 |<--------------------+ | +----+------+ | v | | +--------------------+ | | V v +---+----+ | 2 | AB | | 1 | BA | | 0 | BB | +---+----+
As a fallback this program will use quicksort where the character counts are too high, or if the initial distribution of char[0] is not large enough to break it up into buckets for multithreading.

Tested on a AMD Ryzen 9 7940HS CPU.

// hybrid sort // by c.p.brown 2025 // // radix sort up to n chars, then fall back to quicksort // reverts to mergesort if a bucket distribution of n_procs can't be obtained from char[0] // // TODO: // - radix sort by column, // - multithreading, // - mergesort fallback, // - layered sort using columns in sortby[] struct packet { int[] items; } string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } string printintarray (int[] s, int f) { string[] ii = {"", "\"", "{", "[", "[", "| ", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%d".printf(s[i]) + dd[f]); } o = (o + "%d".printf(s[s.length - 1]) + oo[f]); return o; } string printstringarray (string[] s, int f) { // 0 1 2 3 4 5 6 7 8 string[] ii = {"", "\"", "{", "[", "[", "| ", "", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n", ",\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%s".printf(s[i]) + dd[f]); } o = (o + "%s".printf(s[s.length - 1]) + oo[f]); return o; } string[] readlines (string f) { print("readlines started...\n"); int64 tts = GLib.get_monotonic_time(); string[] o = {}; int maxlines = 0; GLib.File ff = GLib.File.new_for_path(f); if (ff.query_exists()) { GLib.FileStream fstr = GLib.FileStream.open(f,"r"); while (true) { string s = fstr.read_line(); if (s == null) { break; } StringBuilder sbline = new StringBuilder(""); sbline.append(s.strip()); o += strdup(sbline.str); maxlines += 1; } } int64 tte = GLib.get_monotonic_time(); print("read %d lines from %s took %s\n\n".printf(maxlines,f,printusecs(tte-tts))); return o; } string writedatbyindex (string p, string n, string x, int[] so, string[] dat) { if ((dat.length == 0) || (dat[0] == null)) { print("no data to write.\n"); return "no data\n"; } string dirname = p; string filename = n; string extname = x; StringBuilder sb = new StringBuilder(""); if (so.length > 0) { for (int i = 0; i < so.length; i++) { if (so[i] < dat.length) { if (dat[so[i]] != null) { sb.append((dat[so[i]] + "\n")); } } } } else { for (int i = 0; i < dat.length; i++) { sb.append((dat[i] + "\n")); } } if ((sb.str.char_count() != 0) && (filename.char_count() != 0) && (extname.char_count() != 0)) { bool allgood = true; string pth = GLib.Environment.get_current_dir(); if (dirname.index_of("/") == 0) { pth = (pth + "/" + dirname + "/"); } else { pth = dirname + "/"; } GLib.Dir dcr = null; try { dcr = Dir.open (pth, 0); } catch (Error e) { print("WRITE checkdir failed: %s\n",e.message); allgood = false; } File hfile = File.new_for_path(pth.concat(filename,".",extname)); File hdir = File.new_for_path(pth); if (allgood == false) { try { hdir.make_directory_with_parents(); allgood = true; } catch (Error e) { print("WRITE makedirs failed: %s\n",e.message); allgood = false; } } if (allgood) { FileOutputStream hose = hfile.replace(null,false,FileCreateFlags.PRIVATE); try { hose.write(sb.str.data); //print("WRITE written to: %s\n",hfile.get_path()); //print("WRITE ended.\n"); return hfile.get_path(); } catch (Error e) { print("WRITE failed: %s\n",e.message); } } else { print("WRITE couldn't make dir, aborting export.\n"); } } else { print("WRITE empty input, aborting.\n"); } return ""; } void quicksortindexbycol (ref int[] dxs, int c, int cc, int s, int e, string[] dat, ref int th) { int rng = (e - s); if (rng > 0) { int p = s; if (rng > 2) { if (rng > 100) { Random.set_seed(th); p = Random.int_range((s+1),(e-1)); } else { int hh = (rng / 2); if (hh > 0) { p = (s + hh); } } } int i = s; int j = e; int dxp = (dxs[p]*cc) + c; while(i <= j) { while (strcmp(dat[(dxs[i]*cc)+c],dat[dxp]) < 0) { i += 1; } while (strcmp(dat[(dxs[j]*cc)+c],dat[dxp]) > 0) { j -= 1; } if (i <= j) { int t = dxs[i]; dxs[i] = dxs[j]; dxs[j] = t; i += 1; j -= 1; } } if (s < j) { th += 1; quicksortindexbycol(ref dxs,c,cc,s,j,dat,ref th); } if (i < e) { th += 1; quicksortindexbycol(ref dxs,c,cc,i,e,dat,ref th); } } } void rradixmemystrings (int p, ref packet[] asciitable, string[] dat, int[] dupes, int[] dcc, int t, ref int th) { //if (th > 100000) { return; } int rxth = 5; // char count threshhold for radix if (p > 0) { int[] sames = {}; packet[] myasciitable = {}; for (int i = 0; i < 128; i++) { myasciitable += new packet(); } // get lines of length less than p int mxcc = 0; foreach (int d in dupes) { int cc = dcc[d]; mxcc = int.max(cc,mxcc); if (p >= cc) { sames += d; } } // distribute dupes on the ascii table by char p int atv = 0; if (sames.length < dupes.length) { foreach (int d in dupes) { int cc = dcc[d]; if (p < cc) { int q = ((int) dat[d].get_char(dat[d].index_of_nth_char(p))); if (q < 127) { myasciitable[q].items += d; } else { myasciitable[127].items += d; } } } for (int i = 0; i < myasciitable.length; i++) { if (myasciitable[i].items.length > 1) { atv += 1; } } // process dupes for (int i = 0; i < myasciitable.length; i++) { if (myasciitable[i].items.length > 1) { int x = p + 1; th += 1; if ( (i < 127) && (x < 3) && (mxcc < rxth) ) { rradixmemystrings(x, ref myasciitable, dat, myasciitable[i].items, dcc, i, ref th); } else { quicksortindexbycol(ref myasciitable[i].items, 0, 1, 0, (myasciitable[i].items.length - 1), dat, ref th); } } } // backwash asciitable[t].items.resize(0); for (int i = 0; i < sames.length; i++) { asciitable[t].items += sames[i]; } for (int i = 0; i < myasciitable.length; i++) { if (myasciitable[i].items.length > 0) { for (int q = 0; q < myasciitable[i].items.length; q++) { asciitable[t].items += myasciitable[i].items[q]; } } } } } else { // initialize 1st pass int atv = 0; int[] adcc = {}; for (int i = 0; i < dat.length; i++) { adcc += dat[i].char_count(); int q = ((int) dat[i].get_char(dat[i].index_of_nth_char(0))); if (q < 127) { asciitable[q].items += i; } else { asciitable[127].items += i; } } for (int i = 0; i < asciitable.length; i++) { if (asciitable[i].items.length > 1) { atv += 1; } } // bail if distribution is poor, handle using a different technique outside of this function if ((atv < 16) && (dat.length > 5)) { print("inital distribution is %d chars, aborting...\n",atv); asciitable.resize(0); } else { // print distribution of lines by char 0 bool doreport = true; int[] xcc = {}; if (doreport) { int mw = 0; int[] itw = {}; for (int tt = 0; tt < asciitable.length; tt++) { itw += asciitable[tt].items.length; int xc = 0; for (int g = 0; g < asciitable[tt].items.length; g++) { xc = int.max(adcc[asciitable[tt].items[g]],xc); } xcc += xc; if (asciitable[tt].items.length > 0) { mw = int.max(mw,asciitable[tt].items.length); } } print("\ndistribution of lines:\n"); print("asciitable id char | maxc | rows |\n"); print("---------------------+------+--------+\n"); for (int tt = 0; tt < asciitable.length; tt++) { if (itw[tt] > 0) { int sp = 0; string cs = ((char) tt).to_string(); if (tt > 31) { sp = 2; } if (tt == 0) { sp = 2; } if (tt == 127) { sp = 0; } if (tt == 9) { cs = "\\t"; } if (tt == 10) { cs = "\\n"; } print("asciitable[%03d] (%*s) | %04d | %06d | ",tt,sp,cs,xcc[tt],itw[tt]); int bw = ((int) (((itw[tt] * 1.0) / (mw * 1.0)) * 40.0)); if ((bw == 0) && (itw[tt] > 0)) { bw = 1; } for (int k = 0; k < (bw); k++) { print("█"); } if(tt == (asciitable.length - 1)) { print(" <-- unicodes go here"); } print("\n"); } } print("\nintital allocation of row indices for char 0 complete, sorting...\n"); } else { for (int tt = 0; tt < asciitable.length; tt++) { int xc = 0; for (int g = 0; g < asciitable[tt].items.length; g++) { xc = int.max(adcc[asciitable[tt].items[g]],xc); } xcc += xc; } } // begin recursion int x = p + 1; th += 1; for (int tt = 0; tt < asciitable.length; tt++) { if (asciitable[tt].items.length > 1) { // continue radix sorting short ascii strings if ((atv < 6) || ((xcc[tt] < rxth) && (tt < 127))) { print("inital distribution is %d chars\n",atv); rradixmemystrings(x, ref asciitable, dat, asciitable[tt].items, adcc, tt, ref th); } else { // otherwise quicksort quicksortindexbycol(ref asciitable[tt].items, 0, 1, 0, (asciitable[tt].items.length - 1), dat, ref th); } } } } } } void main() { GLib.Intl.setlocale(ALL,""); packet[] asciitable = {}; for (int i = 0; i < 128; i++) { asciitable += new packet(); } string[] s = {}; //s = {"CDF","","BACCDF","CFEGH","CDF","CFEGA","EA","","AEF","EFB","BACC"}; //s = {"BDF","","BACCDF","BFEGH","BDF","BFEGA","BA","","BEF","BFB","BACC"}; //print("s : %s\n",printstringarray(s,2)); s = readlines("./leipzig1M.txt"); //s = readlines("./shellsort_test.txt"); //s = readlines("./hustlermowers_com_au_dealerlocator.txt"); //s = readlines("./xopnet.vala"); //s = readlines("./org_tables.txt"); //s = readlines("./cpbrown_notes.org"); //s = readlines("./ACT_ADDRESS_SITE_psv.psv"); int dth = 0; int[] nop = new int[0]; int64 tts = GLib.get_monotonic_time(); int[] dxs = {}; rradixmemystrings(0, ref asciitable, s, nop, nop, 0, ref dth); if (asciitable.length == 0) { for (int i = 0; i < s.length; i++) { dxs += i; } print("quicksorting %d strings in non partitioned array...\n",dxs.length); quicksortindexbycol(ref dxs, 0, 1, 0, (dxs.length - 1), s, ref dth); print("quicksorting non partitioned data complete.\n"); } else { for (int i = 0; i < asciitable.length; i++) { for (int q = 0; q < asciitable[i].items.length; q++) { dxs += asciitable[i].items[q]; } } } int64 tte = GLib.get_monotonic_time(); print("hybrid radix/quick sort of %d strings took %s\n\n".printf(s.length,printusecs(tte-tts))); print("total recursions is %d\n\n",dth); //for (int i = 0; i < asciitable.length; i++) { // if (asciitable[i].items.length > 0) { //print("[%03d] %s\n",i,printintarray(asciitable[i].items,2)); //print("asciitable[%03d] (%s) has %d rows\n",i,((char) i).to_string(),asciitable[i].items.length); // } //} // verify int allgood = 1; for (int i = 0; i < (dxs.length - 1); i++) { if (strcmp(s[dxs[i]],s[dxs[i+1]]) > 0) { print("text is sorted incorrectly at line %d :\n%s\nis greater than\n%s\n\n",i,s[dxs[i]],s[dxs[i+1]]); allgood = -1; break; } } if (allgood == 1) { //print("last line to be written is : %s\n",s[dxs[(sortorder.length - 1)]]); print("text is sorted correctly, writing to file...\n"); string oup = writedatbyindex("./","rradixsort_test","txt",dxs,s); } }
#+RESULTS:
Compilation succeeded - 6 warning(s) readlines started... read 1000000 lines from ./leipzig1M.txt took 1.29 s distribution of lines: asciitable id char | maxc | rows | ---------------------+------+--------+ asciitable[034] ( ") | 0255 | 075798 | ████████████ asciitable[036] ( $) | 0109 | 000001 | █ asciitable[037] ( %) | 0143 | 000016 | █ asciitable[039] ( ') | 0253 | 006922 | █ asciitable[040] ( () | 0255 | 000953 | █ asciitable[046] ( .) | 0122 | 000002 | █ asciitable[048] ( 0) | 0047 | 000003 | █ asciitable[049] ( 1) | 0197 | 000064 | █ asciitable[050] ( 2) | 0190 | 000008 | █ asciitable[051] ( 3) | 0197 | 000022 | █ asciitable[052] ( 4) | 0100 | 000007 | █ asciitable[053] ( 5) | 0098 | 000006 | █ asciitable[054] ( 6) | 0110 | 000006 | █ asciitable[056] ( 8) | 0032 | 000001 | █ asciitable[057] ( 9) | 0212 | 000027 | █ asciitable[063] ( ?) | 0029 | 000001 | █ asciitable[065] ( A) | 0255 | 096447 | ███████████████ asciitable[066] ( B) | 0255 | 062289 | ██████████ asciitable[067] ( C) | 0255 | 024990 | ████ asciitable[068] ( D) | 0255 | 018391 | ███ asciitable[069] ( E) | 0255 | 016243 | ██ asciitable[070] ( F) | 0255 | 024315 | ████ asciitable[071] ( G) | 0255 | 012182 | ██ asciitable[072] ( H) | 0255 | 054333 | ████████ asciitable[073] ( I) | 0255 | 091341 | ███████████████ asciitable[074] ( J) | 0255 | 008732 | █ asciitable[075] ( K) | 0255 | 004951 | █ asciitable[076] ( L) | 0255 | 016035 | ██ asciitable[077] ( M) | 0255 | 051554 | ████████ asciitable[078] ( N) | 0255 | 019091 | ███ asciitable[079] ( O) | 0255 | 027153 | ████ asciitable[080] ( P) | 0255 | 023311 | ███ asciitable[081] ( Q) | 0255 | 000885 | █ asciitable[082] ( R) | 0255 | 015950 | ██ asciitable[083] ( S) | 0255 | 053977 | ████████ asciitable[084] ( T) | 0255 | 242877 | ████████████████████████████████████████ asciitable[085] ( U) | 0255 | 010388 | █ asciitable[086] ( V) | 0255 | 004054 | █ asciitable[087] ( W) | 0255 | 030221 | ████ asciitable[088] ( X) | 0252 | 000192 | █ asciitable[089] ( Y) | 0255 | 005479 | █ asciitable[090] ( Z) | 0251 | 000735 | █ asciitable[096] ( `) | 0236 | 000040 | █ asciitable[115] ( s) | 0165 | 000007 | █ intital allocation of row indices for char 0 complete, sorting... hybrid radix/quick sort of 1000000 strings took 721.92 ms total recursions is 938301 text is sorted correctly, writing to file...
READLINE:vala:

Read a particular line from a huge file.
Pre-index the lines to allow multithreading.
Requested line number is 1-based.

// read pre-determined lines from a pre-indexed file // by cpbrown 2024 GLib.Mutex mutex; struct packet { public uint[] items; } string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } packet[] packseq ( uint[] x, uint c, bool q, int y, int z, int u) { //print("packseq started...\n"); // x = items to package // q = use items if true, use items count if false, eg {2,12,34,43} vs {0,1,2,3} // y = max packets, usually thread-count // z = nth item to package, eg {0,1,2,3,4,5} nth 2 is {0,2,4} // u = offset items, eg {0,1,2,3,4,5} offset 2 is {2,3,4,5}, {0,1,2,3,4,5} nth 2 offset 1 is {1,3,5} packet[] o = new packet[0]; uint[] srcitems = x; int maxpackets = y; int nthsrc = z; // nth int srcoffset = u; // offset uint srclen = c; uint[] sampleditems = {}; if (q) { srclen = srcitems.length; for (uint j = srcoffset; j < srclen; j += nthsrc) { sampleditems += srcitems[j]; } } else { for (uint j = srcoffset; j < srclen; j += nthsrc) { sampleditems += j; } } //print("sampleditems.length is %d, maxpackets is %d\n",sampleditems.length,maxpackets); if (sampleditems.length > maxpackets) { // y packets of n items uint itemsperpacket = (sampleditems.length / maxpackets); double neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); uint mxn = 0; //print("itemsperpacket is %u, neededpackets is %.1f\n",itemsperpacket,neededpackets); while (neededpackets > maxpackets) { if (mxn > 1000000) { //print("max packet count of 1 million packets reached, bailing...\n"); return o; } itemsperpacket += 1; neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); mxn += 1; } //print("itemsperpacket is %u, neededpackets is %.1f\n",itemsperpacket,neededpackets); uint np = (uint) neededpackets; uint dregs = sampleditems.length - (itemsperpacket * ((uint) neededpackets)); uint h = 0; if (dregs > 0) { h = 1; } if ((np + h) > 0) { o = new packet[(np+h)]; uint f = 0; for (uint i = 0; i < np; i++) { o[i] = new packet(); for (uint t = 0; t < itemsperpacket; t++) { o[i].items += sampleditems[f]; f += 1; } } if (dregs > 0) { o[np] = new packet(); for (uint t = 0; t < dregs; t++) { o[np].items += sampleditems[f]; f += 1; } } } } else { // one item per packet for (uint k = 0; k < sampleditems.length; k++) { packet oo = new packet(); oo.items += sampleditems[k]; o += oo; } } //print("packseq complete.\n"); return o; } class gnafdetailsearch : Object { private uint[] chunks; private uint thispacket; private string[] addresscodes; private string[] loccodes; private string postcode; private string streetnum; private string detailsfilepath; public gnafdetailsearch ( uint pak, uint[] chu, string[] ady, string[] loc, string poc, string num, string det ) { thispacket = pak; chunks = chu; addresscodes = ady; loccodes = loc; postcode = poc; streetnum = num; detailsfilepath = det; } public void eval (ref string[] geocodes, ref int[] threadcheck, ref int weredonehere, int64[] bidx) { foreach (uint x in chunks) { if (weredonehere > 0) { break; } GLib.File addressfile = GLib.File.new_for_path(detailsfilepath); if (addressfile.query_exists()) { GLib.FileStream dfstr = GLib.FileStream.open(detailsfilepath,"r"); StringBuilder dline = new StringBuilder(""); dfstr.seek(((long) bidx[x]),SET); string drl = dfstr.read_line(); if (drl != null) { dline.append(drl.strip()); if (dline.str.char_count() > 0) { string[] dcols = dline.str.split("|"); for (int n = 0; n < addresscodes.length; n++) { int ii = dline.str.index_of(addresscodes[n]); if (ii != -1) { int qq = dline.str.index_of(loccodes[n]); if (qq != -1) { int oo = dline.str.index_of(postcode); if (oo != -1) { bool gg = ((strcmp(dcols[6],streetnum) == 0) || (strcmp(dcols[17],streetnum) == 0)); if (gg) { int rr = dline.str.index_of("GA"); if (rr != -1) { //rr += 1; int hh = dline.str.index_of("|",rr); if (hh != -1) { StringBuilder adcode = new StringBuilder(""); adcode.append(dline.str.substring(rr,(hh - rr))); mutex.lock(); geocodes[thispacket] = adcode.str; weredonehere = 1; mutex.unlock(); break; //print("found possible match: %s\n",geocodes[geocodes.length - 1]); } } } } } } } } } } } threadcheck[thispacket] = 1; } } void main() { mutex = GLib.Mutex(); int nthr = ((int) GLib.get_num_processors()); int64[] bidx = {}; uint g = 1000000; string psvdir = "./psv/"; string detailsfilename = "NSW_ADDRESS_DETAIL_psv.psv"; GLib.File detailsfile = GLib.File.new_for_path(psvdir + detailsfilename); if (detailsfile.query_exists()) { print("detailsfile found: %s\n",detailsfile.get_path()); GLib.FileStream fstr = GLib.FileStream.open((psvdir + detailsfilename),"r"); StringBuilder sbline = new StringBuilder(""); int64 accumulatedlines = 0; while (true) { string s = fstr.read_line(); if (s == null) { break; } accumulatedlines += (s.length + 1); bidx += accumulatedlines; } print("%s has %d lines\n",detailsfilename,bidx.length); GLib.FileStream sstr = GLib.FileStream.open((psvdir + detailsfilename),"r"); sstr.seek(((long) bidx[g-1]),SET); string s = sstr.read_line(); if (s != null) { sbline.append(s); } if (sbline.str.char_count() > 0) { print("index check: line 1000000 is %s\n",sbline.str.strip()); } string[] addresscodes = {"SW2880444","SW2880461","SW2873828","SW2880443","SW327749","SW2880450","SW2880438","SW2880437","SW2880447","SW2880458","SW3565179","SW2880446","SW2880445","SW2880441","SW2880439","SW2880455","SW2880453","SW2880449","SW3576944","SW2880464","SW2993890","SW2880460","SW2880442","SW2880454","SW2880451","SW2880448","SW2880452","SW2880440","SW3561635","SW2880459","SW2880456","SW2880457"}; string[] loccodes = {"loceaa57fa3272e","locdd9679f44083","loc96da8afc3d2d","loc2fdf87e092bc","loc7b58be2bee5d","loc22a73fe48374","loc04e9f4bdba8c","locc0764149b600","loc505caada930b","loca9e740a3c1e3","locc28a9c36431f","loc5560df1ac440","locef05745fb16d","loc6b9200960de8","loc768adb4ecaa1","loc730f8cb0d761","loc771141cd5b15","locc606180ec0a9","loc52d65ddc6414","locf43b9d9f4240","loc9c0a49900253","locf27d731b0255","loc48d3cabbbc14","loc67395bf5b53d","loc64288deecac1","loc8ec1cb764645","locb0165da80688","loc435dda1f552b","loc7fac84f28841","loc0dafe55d09fe","loc5a4cd11021e1","loce6b9edea9174"}; string postcode = "2750"; string streetnumber = "2"; string detailsfilepath = (psvdir + detailsfilename); print("multithreading search of %d lines...\n",bidx.length); int64 tts = GLib.get_monotonic_time(); uint[] bq = new uint[0]; packet[] pak = packseq(bq,((uint) bidx.length),false,nthr,1,0); print("split indices into %d packets of %d items...\n",pak.length,pak[0].items.length); int[] threadcheck = new int[pak.length]; for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } string[] geocodes = new string[pak.length]; int weredonehere = 0; ThreadPool<gnafdetailsearch> gs = new ThreadPool<gnafdetailsearch>.with_owned_data( (gnafdetailsearch) => {gnafdetailsearch.eval(ref geocodes, ref threadcheck, ref weredonehere, bidx); },nthr,false ); for (uint x = 0; x < pak.length; x++) { gs.add( new gnafdetailsearch( x, pak[x].items, addresscodes, loccodes, postcode, streetnumber, detailsfilepath ) ); } int lastindex = pak.length - 1; while (threadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } int64 tte = GLib.get_monotonic_time(); print("multithreaded search took %s\n",printusecs(tte-tts)); for (int i = 0; i < geocodes.length; i++) { if (geocodes[i] != null) { print("found geocode id %s\n",geocodes[i]); } } } }
#+RESULTS:
Compilation succeeded - 5 warning(s) detailsfile found: /home/cpb/Desktop/txt/source/psv/NSW_ADDRESS_DETAIL_psv.psv NSW_ADDRESS_DETAIL_psv.psv has 5093369 lines index check: line 1000000 is GANSW715681425|2008-04-18|2021-07-07||||1733|||||||||||16|||||NSW2868058||loc91fd57f2790a|P|2330||1733/1062060|2|711245734|7||3048244| multithreading search of 5093369 lines... split indices into 16 packets of 318336 items... multithreaded search took 2.39 s found geocode id GANSW705642589
REDBUTTON:vala:

A button with a signal that changes its color
Try to make this code as simple as possible without golfing.
GTK OOP is great for complex interfaces, onerous for simple ones, and thus a massive hurdle for new users, especially those who were spoiled with things like:

RED [ needs 'View ] View/tight [ panel [ buton 120x30 red "DOIT" [ face/color: green face/text: "DONE" ] ] ]
// gtk ui simplification challenge // make a red button that changes to green when clicked // reduce the code as much as possible without golfing // by c.b.brown 2024 using Gtk; int main(string[] args) { Gtk.Application myapp = new Gtk.Application("com.mytest.mytest",GLib.ApplicationFlags.DEFAULT_FLAGS); myapp.activate.connect (() => { Gtk.ApplicationWindow mywin = new Gtk.ApplicationWindow(myapp); Gtk.Box mybox = new Gtk.Box(HORIZONTAL,0); Gtk.Button mybutton = new Gtk.Button.with_label("DOIT"); mybutton.set_css_classes({"redbutton"}); mybutton.hexpand = true; mybox.append(mybutton); mybutton.clicked.connect(() => { mybutton.set_css_classes({"greenbutton"}); mybutton.label = "DONE"; }); string mycss = """ .redbutton { background: #FF0000; } .greenbutton { background: #00FF00; } """; Gtk.CssProvider mycsp = new Gtk.CssProvider(); mycsp.load_from_string(mycss); Gdk.Display thisdisplay = Gdk.Display.get_default(); Gtk.StyleContext.add_provider_for_display(thisdisplay,mycsp,Gtk.STYLE_PROVIDER_PRIORITY_USER); mywin.set_child(mybox); mywin.default_width = 120; mywin.default_height = 70; mywin.present(); }); return myapp.run(args); }
REPLACECHAR:vala:

Test 3 methods of removing a char:
  • StringBuilder.replace() : approx 5 microseconds
  • manual incramental index_of() & substring() : approx 10 microseconds
  • default string.replace() : approx 35 micorseconds

Not sure how stringBuilder is faster, my guess is its doing in-place char-array shuffling/resizing, while the manual method makes new substrings, then uses them to rebuild the string.
The default replace may be restarting from the beginning of the string after each replace.

Update this later with an attempt at manually manipulating a unichar array.

string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } string removecharuni (string h, unichar b) { int64 tts = GLib.get_monotonic_time(); int hcc = h.char_count(); uint8[] u = h.data; int ul = u.length; string bs = b.to_string(); uint8[] bu = bs.data; int bul = bu.length - 1; int nxt = 0; int ii = 0; print("%s.length in bytes is %d\n",bs,bul); print("string.length in bytes is %d\n",h.length); while (ii != -1) { ii = -1; for (int i = nxt; i < ul; i++) { if (u[i] == bu[0]) { if ((ul - i) >= bul) { int v = 0; for (int q = i; q <= (i + bul); q++) { if (u[q] != bu[v]) { break; } v += 1; } if (v == bul) { ii = i; nxt = i; for (int q = i; q < (u.length - 1); q++) { u[q] = u[q+1]; } u.resize(u.length - bul); break; } } } } } string s = ((string) u); //if (s.validate(s.length,null) == false) { s = s + "\0"; } int64 tte = GLib.get_monotonic_time(); print("\nmanual unichar[] replace took %s\n",printusecs(tte-tts)); return s; } string removechar (string h, unichar b) { int64 tts = GLib.get_monotonic_time(); string s = h; int ii = 0; int nxt = 0; int n = 0; int mx = s.char_count(); string u = b.to_string(); int l = b.to_utf8(u); while (ii != -1) { if (n > mx) { break; } ii = s.index_of_char(b,nxt); if (ii != -1) { //print("\t%s",s.substring(0,ii)); //print("...%s\n",s.substring(ii+l)); s = s.substring(0,ii) + s.substring(ii+l); nxt = ii; } n += 1; } int64 tte = GLib.get_monotonic_time(); print("\nmanual char replace took %s\n",printusecs(tte-tts)); return s; } string removecharsb (string h, unichar b) { int64 tts = GLib.get_monotonic_time(); StringBuilder sb = new StringBuilder(""); sb.append(h); string bs = b.to_string(); sb.replace(bs,"",0); int64 tte = GLib.get_monotonic_time(); print("\nstringbuilder char replace took %s\n",printusecs(tte-tts)); return sb.str; } string removechardefault (string h, unichar b) { int64 tts = GLib.get_monotonic_time(); string s = h; string bs = b.to_string(); s = s.replace(bs,""); int64 tte = GLib.get_monotonic_time(); print("\ndefault char replace took %s\n",printusecs(tte-tts)); return s; } void main() { GLib.Intl.setlocale(ALL,""); string s = "o零ne hund零red b零illion eigh零t hundre零d a零nd fou零rty two零 mill零ion tw零enty th零ree th零ous零and th零ree hu零ndred零 an零d si零xty 零fi零ve"; print("%s\n",s); unichar b = '零'; string q = s; q = removechar(q,b); print("%s\n",q); q = s; q = removecharsb(q,b); print("%s\n",q); q = s; q = removechardefault(q,b); print("%s\n",q); q = s; q = removecharuni(q,b); print("%s\n",q); }
#+RESULTS:
Compilation succeeded - 1 warning(s) o零ne hund零red b零illion eigh零t hundre零d a零nd fou零rty two零 mill零ion tw零enty th零ree th零ous零and th零ree hu零ndred零 an零d si零xty 零fi零ve manual char replace took 11 μs one hundred billion eight hundred and fourty two million twenty three thousand three hundred and sixty five stringbuilder char replace took 4 μs one hundred billion eight hundred and fourty two million twenty three thousand three hundred and sixty five default char replace took 42 μs one hundred billion eight hundred and fourty two million twenty three thousand three hundred and sixty five 零.length in bytes is 2 string.length in bytes is 167 manual unichar[] replace took 5 μs o零ne hund零red b零illion eigh零t hundre零d a零nd fou零rty two零 mill零ion tw零enty th零ree th零ous零and th零ree hu零ndred零 an零d si零xty 零fi零ve
REPLACEUNQUOTED:vala:

Replace text if it isn't quoted.

Includes a bonus attempt at sampling to determine if its worth multithreading, since using ThreadPool can be slower in some cases. Interestingly the average sample time was closest to actual, at 16 tasks 16 threads, as opposed to max and median. Also the difference between sample times can be used to determine the task:thread ratio, where consistently even times can be sent at 1:1, high variance should go at 2:1 or more. There is a point of diminishing returns, but not sure how to determine it. In this case doubling the task count cut the total time by at least 5x.

// replace unquoted strings // ignores escaped quotes // by cpbrown, 2025 // // assumptions: // quotes are pairs, even if the same, eg " & " // quote open preceeds quote close // string doesn't contain delimiters \x1F or \x1E ! (should check 1st) GLib.Mutex mutex; string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } double getmedianintsixtyfour (int64[] n) { int l = n.length; double m = -1.0; if ((l & 1) == 0) { int aa = ((int) (l * 0.5)) - 1; int bb = aa + 1; m = ((n[aa] + n[bb]) * 0.5); } else { int aa = ((int) (l * 0.5)); m = (double) n[aa]; } return m; } double getaverageintsixtyfour (int64[] n) { int64 s = 0; for (int i = 0; i < n.length; i++) { s += n[i]; } if (s == 0) { return 0.0; } return (((double) s) / n.length); } void writestring (string l, string n) { bool allgood = false; var dd = GLib.Environment.get_current_dir(); string nn = n.concat(".txt"); string ff = Path.build_filename (dd, nn); File fff = File.new_for_path (ff); FileOutputStream oo = null; try { oo = fff.replace (null, false, FileCreateFlags.PRIVATE); allgood = true; } catch (Error e) { print ("Error: couldn't make outputstream.\n\t%s\n", e.message); } if (allgood) { try { oo.write (l.data); } catch (Error e) { print ("Error: couldn't write to file.\n\t%s\n", e.message); } } else { print("Error: couldn't open output file for writing\n");} } struct packet { public int[] items; } packet[] packseq (int ind, int[] x, int c, bool q, int y, int z, int u) { // x = items to package // q = use items if true, use items count if false, eg {2,12,34,43} vs {0,1,2,3} // y = max packets, usually thread-count // z = nth item to package, eg {0,1,2,3,4,5} nth 2 is {0,2,4} // u = offset items, eg {0,1,2,3,4,5} offset 2 is {2,3,4,5}, {0,1,2,3,4,5} nth 2 offset 1 is {1,3,5} packet[] o = new packet[0]; int[] srcitems = x; int maxpackets = y; int nthsrc = z; // nth int srcoffset = u; // offset int srclen = c; int[] sampleditems = {}; if (q) { srclen = srcitems.length; for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += srcitems[j]; } } else { for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += j; } } if (sampleditems.length > y) { // y packets of n items int itemsperpacket = (sampleditems.length / maxpackets); double neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); int mxn = 0; while (neededpackets > maxpackets) { if (mxn > 1000000) { return o; } itemsperpacket += 1; neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); mxn += 1; } int np = (int) neededpackets; int dregs = sampleditems.length - (itemsperpacket * ((int) neededpackets)); int h = 0; if (dregs > 0) { h = 1; } if ((np + h) > 0) { o = new packet[(np+h)]; int f = 0; for (int i = 0; i < np; i++) { o[i] = new packet(); for (int t = 0; t < itemsperpacket; t++) { o[i].items += sampleditems[f]; f += 1; } } if (dregs > 0) { o[np] = new packet(); for (int t = 0; t < dregs; t++) { o[np].items += sampleditems[f]; f += 1; } } } } else { // one item per packet for (int k = 0; k < sampleditems.length; k++) { packet oo = new packet(); oo.items += sampleditems[k]; o += oo; } } return o; } public class qreplacer { private string r; private string w; private string e; private string qo; private string qc; private int[] chunq; private int packetnum; public qreplacer (int pkn, string rs, string ws, string qs, string qe, string es, int[] chu) { r = rs; w = ws; e = es; qo = qs; qc = qe; chunq = chu; packetnum = pkn; } public void run (string[] segs, ref int packetscomplete) { foreach (int x in chunq) { StringBuilder xs = new StringBuilder(""); xs.append(segs[x]); if (xs.str.index_of(r) == -1) { continue; } if (xs.str.index_of(qo) != -1) { string eqd = "%c".printf('\x1E'); string qd = "%c".printf('\x1F'); int ofs = 0; string[] qq = {qo,qc}; bool ispair = true; if (xs.str.index_of(qc) == -1) { qq[1] = qo; ispair = false; } xs.replace((e+qq[0]),eqd,0); if (ispair) { xs.replace((e+qq[1]),eqd,0); } xs.replace(qq[0],qd,0); if (ispair) { xs.replace(qq[1],qd,0); } if (xs.str.index_of(qd) == 0) { ofs = 1; } string[] uq = xs.str.split(qd); for (int i = ofs; i < uq.length; i += 2) { uq[i] = uq[i].replace(" ",w); } ofs = (ofs - 1).abs(); for (int i = ofs; i < uq.length; i += 2) { uq[i] = qq[0] + uq[i] + qq[1]; } xs.erase(); xs.insert(0,string.joinv("",uq)); int ii = 0; int v = 0; int n = 0; while (ii != -1) { if (n > 100000) { break; } ii = xs.str.index_of(eqd); if (ii != -1) { xs.erase(ii,eqd.length); xs.insert(ii,(e+qq[v % 2])); v += 1; } n += 1; } } else { xs.replace(r,w,0); } segs[x] = xs.str; } mutex.lock(); packetscomplete += 1; mutex.unlock(); } } string qreplace (string s, string rs, string ws, string qo, string qc, string es) { //int64 tts = GLib.get_monotonic_time(); string r = rs; string w = ws; string e = es; StringBuilder xs = new StringBuilder(s); if (xs.str.index_of(r) == -1) { //int64 tte = GLib.get_monotonic_time(); //print("qreplace took %.4f μs\n",((double) (tte - tts))); return s; } if (s.index_of(qo) != -1) { string eqd = "%c".printf('\x1E'); string qd = "%c".printf('\x1F'); int ofs = 0; string[] qq = {qo,qc}; bool ispair = true; if (xs.str.index_of(qc) == -1) { qq[1] = qo; ispair = false; } xs.replace((e+qq[0]),eqd,0); if (ispair) { xs.replace((e+qq[1]),eqd,0); } xs.replace(qq[0],qd,0); if (ispair) { xs.replace(qq[1],qd,0); } if (xs.str.index_of(qd) == 0) { ofs = 1; } string[] uq = xs.str.split(qd); for (int i = ofs; i < uq.length; i += 2) { uq[i] = uq[i].replace(" ",w); } ofs = (ofs - 1).abs(); for (int i = ofs; i < uq.length; i += 2) { uq[i] = qq[0] + uq[i] + qq[1]; } xs.erase(); xs.insert(0,string.joinv("",uq)); int ii = 0; int x = 0; int n = 0; while (ii != -1) { if (n > 100000) { break; } ii = xs.str.index_of(eqd); if (ii != -1) { xs.erase(ii,eqd.length); xs.insert(ii,(e+qq[x % 2])); x += 1; } n += 1; } } else { xs.replace(r,w,0); } //int64 tte = GLib.get_monotonic_time(); //print("qreplace took %s\n",printusecs(tte - tts)); return(xs.str); } void qreplaceinplace (ref StringBuilder xs, string rs, string ws, string qo, string qc, string es) { //int64 tts = GLib.get_monotonic_time(); string r = rs; string w = ws; string e = es; //StringBuilder xs = new StringBuilder(s); if (xs.str.index_of(r) == -1) { //int64 tte = GLib.get_monotonic_time(); //print("qreplace took %.4f μs\n",((double) (tte - tts))); return; } if (xs.str.index_of(qo) != -1) { string eqd = "%c".printf('\x1E'); string qd = "%c".printf('\x1F'); int ofs = 0; string[] qq = {qo,qc}; bool ispair = true; if (xs.str.index_of(qc) == -1) { qq[1] = qo; ispair = false; } xs.replace((e+qq[0]),eqd,0); if (ispair) { xs.replace((e+qq[1]),eqd,0); } xs.replace(qq[0],qd,0); if (ispair) { xs.replace(qq[1],qd,0); } if (xs.str.index_of(qd) == 0) { ofs = 1; } string[] uq = xs.str.split(qd); for (int i = ofs; i < uq.length; i += 2) { uq[i] = uq[i].replace(" ",w); } ofs = (ofs - 1).abs(); for (int i = ofs; i < uq.length; i += 2) { uq[i] = qq[0] + uq[i] + qq[1]; } xs.erase(); xs.insert(0,string.joinv("",uq)); int ii = 0; int x = 0; int n = 0; while (ii != -1) { if (n > 100000) { break; } ii = xs.str.index_of(eqd); if (ii != -1) { xs.erase(ii,eqd.length); xs.insert(ii,(e+qq[x % 2])); x += 1; } n += 1; } } else { xs.replace(r,w,0); } //int64 tte = GLib.get_monotonic_time(); //print("qreplace took %s\n",printusecs(tte - tts)); //return(xs.str); } public class refsplitstring { public string[] str; public refsplitstring (string s, string d) { str = s.split(d); } } void main() { GLib.Intl.setlocale(ALL,""); mutex = GLib.Mutex(); int nthr = (int) GLib.get_num_processors(); string s = "a \\{ \\} string {goes \\{here \\} and} not { here}"; string r = qreplace (s," ","\n","{","}","\\"); // | | | | | // find + | with quotes escape // replace + // desired result: // a // \{ // \} // string // {goes \{here \} and} // not // { here} //print(r); StringBuilder sorg = new StringBuilder(""); //string testfile = "cpbrown_notes.org"; string testfile = "leipzig1M.txt"; int64 ltts = GLib.get_monotonic_time(); string ff = Path.build_filename ("./", testfile); GLib.File og = File.new_for_path(ff); if (og.query_exists() == true) { try { uint8[] ca; string oe; og.load_contents (null, out ca, out oe); sorg.append((string) ca); } catch (Error e) { print ("\tfailed to read %s: %s\n", og.get_path(), e.message); } } int64 ltte = GLib.get_monotonic_time(); print("file load took %s\n",printusecs(ltte - ltts)); int64 ctts = GLib.get_monotonic_time(); int xn = 0; int xnn = 0; int xcc = -1; while (xnn != -1) { xcc += 1; xnn = sorg.str.index_of_char('\n',xn); xn = (xnn + 1); } int64 ctte = GLib.get_monotonic_time(); print("line-count took %s\n",printusecs(ctte - ctts)); print("\n\ncheck times for multithreading a %d-line file:\n",xcc); string[] segs = sorg.str.split("\'"); for (int i = 0; i < segs.length; i++) { segs[i] = segs[i] + "\'"; } print("%s fills %d segments split at\"\'\"\n",testfile,segs.length); int64 stts = GLib.get_monotonic_time(); int samples = int.min(nthr,int.max(1,segs.length)); int64 maxqtime = 0; uint rr = GLib.Random.int_range(0,segs.length); int64[] sampletimes = {}; for (int i = 0; i < samples; i++) { print("\tsampling seg[%u]... ",rr); int64 tts = GLib.get_monotonic_time(); r = qreplace(segs[rr],"-","","\'","\'","\\"); int64 tte = GLib.get_monotonic_time(); print("%s\n",printusecs(tte-tts)); sampletimes += (tte - tts); maxqtime = int64.max(maxqtime,(tte-tts)); GLib.Random.set_seed(i+1); rr = GLib.Random.int_range(0,segs.length); } int64 stte = GLib.get_monotonic_time(); print("sample: took %s\n",printusecs(stte - stts)); int64 msf = (int64) getmedianintsixtyfour(sampletimes); int64 asf = (int64) getaverageintsixtyfour(sampletimes); print("sample: qreplace max sample time was %s\n",printusecs(maxqtime)); print("sample: qreplace median sample time was %s\n",printusecs(msf)); print("sample: qreplace average sample time was %s\n\n",printusecs(asf)); print("guesstimate: single thread: max time * segs.length is %s\n",printusecs(maxqtime * segs.length)); print("guesstimate: single thread: median time * segs.length is %s\n",printusecs(msf * segs.length)); print("guesstimate: single thread: average time * segs.length is %s\n\n",printusecs(asf * segs.length)); print( "guesstimate: multi-thread: ((max time * segs.length) / %d) is %s\n", nthr, printusecs( ((maxqtime * segs.length) / nthr) ) ); print( "guesstimate: multi-thread: ((median time * segs.length) / %d) is %s\n", nthr, printusecs( ((msf * segs.length) / nthr) ) ); print( "guesstimate: multi-thread: ((average time * segs.length) / %d) is %s\n\n", nthr, printusecs( ((asf * segs.length) / nthr) ) ); int64 tts = GLib.get_monotonic_time(); r = qreplace(sorg.str,"-","","\'","\'","\\"); int64 tte = GLib.get_monotonic_time(); print("actual: qreplace single-threaded time was %s\n",printusecs(tte - tts)); //writestring(r,"qreplacetest"); tts = GLib.get_monotonic_time(); qreplaceinplace(ref sorg,"-","","\'","\'","\\"); tte = GLib.get_monotonic_time(); print("actual: qreplaceinplace single-threaded time was %s\n",printusecs(tte - tts)); //writestring(sorg.str,"qreplaceinplacetest"); // threaded test int64 mtts = GLib.get_monotonic_time(); int[] segcounts = new int[0]; packet[] pak = packseq(0,segcounts,segs.length,false,nthr,1,0); int packetscomplete = 0; ThreadPool<qreplacer> qrt = new ThreadPool<qreplacer>.with_owned_data((qreplacer) => {qreplacer.run(segs, ref packetscomplete);},nthr,false); for (int i = 0; i < pak.length; i++) { qrt.add( new qreplacer(i,"-","","\'","\'","\\",pak[i].items)); } // wait for threads to complete... while (packetscomplete != pak.length) { GLib.Thread.usleep(1); } int64 mtte = GLib.get_monotonic_time(); print("actual: qreplace multi-threaded (%d tasks, %d threads) time was %s\n",pak.length,nthr,printusecs(mtte - mtts)); // thread test at cpu threads * 2 int64 mxtts = GLib.get_monotonic_time(); packet[] pakb = packseq(0,segcounts,segs.length,false,(nthr * 2),1,0); packetscomplete = 0; ThreadPool<qreplacer> qrtb = new ThreadPool<qreplacer>.with_owned_data((qreplacer) => {qreplacer.run(segs, ref packetscomplete);},nthr,false); for (int i = 0; i < pakb.length; i++) { qrtb.add( new qreplacer(i,"-","","\'","\'","\\",pakb[i].items)); } // wait for threads to complete... while (packetscomplete != pakb.length) { GLib.Thread.usleep(1); } int64 mxtte = GLib.get_monotonic_time(); print("actual: qreplace multi-threaded (%d tasks, %d threads) time was %s\n",pakb.length,nthr,printusecs(mxtte - mxtts)); // thread test at cpu threads * 4 int64 mxxtts = GLib.get_monotonic_time(); packet[] pakc = packseq(0,segcounts,segs.length,false,(nthr * 4),1,0); packetscomplete = 0; ThreadPool<qreplacer> qxrtb = new ThreadPool<qreplacer>.with_owned_data((qreplacer) => {qreplacer.run(segs, ref packetscomplete);},nthr,false); for (int i = 0; i < pakc.length; i++) { qxrtb.add( new qreplacer(i,"-","","\'","\'","\\",pakc[i].items)); } // wait for threads to complete... while (packetscomplete != pakc.length) { GLib.Thread.usleep(1); } int64 mxxtte = GLib.get_monotonic_time(); print("actual: qreplace multi-threaded (%d tasks, %d threads) time was %s\n",pakc.length,nthr,printusecs(mxxtte - mxxtts)); //sorg.erase(0,-1); //print("sorg erased to %d char_count\n",sorg.str.char_count()); //sorg.append(string.joinv("",segs)); //writestring(segs[0],"qreplacetest"); }
#+RESULTS:
Compilation succeeded - 10 warning(s) file load took 110.83 ms line-count took 15.61 ms check times for multithreading a 1000000-line file: leipzig1M.txt fills 357829 segments split at"'" sampling seg[281619]... 55 μs sampling seg[161700]... 1 μs sampling seg[64691]... 15 μs sampling seg[51467]... 0 μs sampling seg[40327]... 24 μs sampling seg[196955]... 1 μs sampling seg[309566]... 0 μs sampling seg[328080]... 0 μs sampling seg[229316]... 4 μs sampling seg[185874]... 1 μs sampling seg[16055]... 0 μs sampling seg[268314]... 1 μs sampling seg[140713]... 1 μs sampling seg[230532]... 11 μs sampling seg[280563]... 1 μs sampling seg[82380]... 0 μs sample: took 208 μs sample: qreplace max sample time was 55 μs sample: qreplace median sample time was 2 μs sample: qreplace average sample time was 7 μs guesstimate: single thread: max time * segs.length is 19.68 s guesstimate: single thread: median time * segs.length is 715.66 ms guesstimate: single thread: average time * segs.length is 2.50 s guesstimate: multi-thread: ((max time * segs.length) / 16) is 1.23 s guesstimate: multi-thread: ((median time * segs.length) / 16) is 44.73 ms guesstimate: multi-thread: ((average time * segs.length) / 16) is 156.55 ms actual: qreplace single-threaded time was 2.54 s actual: qreplaceinplace single-threaded time was 2.46 s actual: qreplace multi-threaded (16 tasks, 16 threads) time was 474.16 ms actual: qreplace multi-threaded (32 tasks, 16 threads) time was 67.89 ms actual: qreplace multi-threaded (64 tasks, 16 threads) time was 68.31 ms
RESIZE:vala:

Check array resizing.

  • resizing a fixed-length array results in an overflow warning
  • resizing a variable-length array does not

The overflow warning was:
‘memset’ specified size 18446744073709551568 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]

Fixed length array stresstest was also approx 10% slower than the variable array.

string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.3f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.3f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.3f m".printf((((double) u) / 60000000.0)); } return us; } void snortintsixtyfour (int64[] l) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; uint n = l.length; foreach (int g in gaps) { for (uint i = 0; i < n; i++) { int64 t = l[i]; uint j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; j -= g; } l[j] = t; } } } double medianmyintsixtyfour (int64[] n) { uint l = n.length; double m = -1.0; if ((l & 1) == 0) { uint aa = ((uint) (l * 0.5)) - 1; uint bb = aa + 1; m = ((n[aa] + n[bb]) * 0.5); } else { uint aa = ((uint) (l * 0.5)); m = (double) n[aa]; } return m; } void main () { string[] s = {"zero","one","two","three","four","five","six","seven","eight","nine","ten"}; print("s = {%s}\n",string.joinv(",",s)); s.resize(5); print("s resized to %d = {%s}\n",s.length,string.joinv(",",s)); s.resize(11); print("s resized back to %d = {%s}\n",s.length,string.joinv(",",s)); for (int i = 0; i < s.length; i++) { if (s[i] != null) { if (s[i].validate()) { print("s[%d] \"%s\" is valid\n",i,s[i]); } else { print("s[%d] is not valid\n",i); } } else { print("s[%d] is null\n",i); } } s = {"zero","one","two","three","four","five","six","seven","eight","nine","ten"}; string[] d = {}; for (int i = 0; i < s.length; i++) { d += s[i]; } print("\nd = {%s}\n",string.joinv(",",d)); d.resize(5); print("d resized to %d = {%s}\n",d.length,string.joinv(",",d)); d.resize(11); print("d resized back to %d = {%s}\n",d.length,string.joinv(",",d)); for (int i = 0; i < d.length; i++) { if (d[i] != null) { if (d[i].validate()) { print("d[%d] \"%s\" is valid\n",i,d[i]); } else { print("d[%d] is not valid\n",i); } } else { print("d[%d] is null\n",i); } } print("\n"); int64[] fixedarraytimes = {}; int64[] variablearraytimes = {}; for (int j = 0; j < 10; j++) { int64 tts = GLib.get_monotonic_time(); uint[] p = new uint[10000000]; for (uint i = 0; i < p.length; i++) { p[i] = i; } uint[] q = new uint[5000000]; for (uint i = 0; i < q.length; i++) { q[i] = p[i]; } int64 tte = GLib.get_monotonic_time(); fixedarraytimes += (tte - tts); int64 vts = GLib.get_monotonic_time(); uint[] r = {}; for (uint i = 0; i < 10000000; i++) { r += i; } r.resize(5000000); int64 vte = GLib.get_monotonic_time(); variablearraytimes += (vte - vts); } snortintsixtyfour(fixedarraytimes); snortintsixtyfour(variablearraytimes); int64 medianfixedtime = (int64) medianmyintsixtyfour(fixedarraytimes); int64 medianvariabletime = (int64) medianmyintsixtyfour(variablearraytimes); print("fastest fixed uint[] test was %s\n",printusecs(fixedarraytimes[0])); print("slowest fixed uint[] test was %s\n",printusecs(fixedarraytimes[(fixedarraytimes.length - 1)])); print("median fixed uint[] test @ 10x took %s\n",printusecs(medianfixedtime)); print("\nfastest variable uint[] test was %s\n",printusecs(variablearraytimes[0])); print("slowest variable uint[] test was %s\n",printusecs(variablearraytimes[(variablearraytimes.length - 1)])); print("meidan variable uint[] test @ 10x took %s\n",printusecs(medianvariabletime)); double perc = (1.0 - (((double) medianfixedtime) / ((double) medianvariabletime))) * 100.0; string sdiff = "faster"; if (perc < 0.0) { sdiff = "slower"; } print("\nfixed uint[] was %.3f%% %s than variable uint[]\n",perc.abs(),sdiff); perc = (1.0 - (((double) medianvariabletime) / ((double) medianfixedtime))) * 100.0; sdiff = "faster"; if (perc < 0.0) { sdiff = "slower"; } print("variable uint[] was %.3f%% %s than fixed uint[]\n",perc.abs(),sdiff); }
#+RESULTS:
s = {zero,one,two,three,four,five,six,seven,eight,nine,ten} s resized to 5 = {zero,one,two,three,four} s resized back to 11 = {zero,one,two,three,four,,,,,,} s[0] "zero" is valid s[1] "one" is valid s[2] "two" is valid s[3] "three" is valid s[4] "four" is valid s[5] is null s[6] is null s[7] is null s[8] is null s[9] is null s[10] is null d = {zero,one,two,three,four,five,six,seven,eight,nine,ten} d resized to 5 = {zero,one,two,three,four} d resized back to 11 = {zero,one,two,three,four,,,,,,} d[0] "zero" is valid d[1] "one" is valid d[2] "two" is valid d[3] "three" is valid d[4] "four" is valid d[5] is null d[6] is null d[7] is null d[8] is null d[9] is null d[10] is null fastest fixed uint[] test was 43.174 ms slowest fixed uint[] test was 73.919 ms median fixed uint[] test @ 10x took 46.112 ms fastest variable uint[] test was 38.765 ms slowest variable uint[] test was 66.573 ms meidan variable uint[] test @ 10x took 42.297 ms fixed uint[] was 9.020% slower than variable uint[] variable uint[] was 8.273% faster than fixed uint[]
ROWCTRL:vala:

Misc row manipulation functions

// row manipulation tests, // oop dresscode is for easier integration with another project // by c.p.brown 2024 void snortint (int[] l, bool r) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = l.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int j = i; if (j >= g) { bool c = (l[j - g] > t); if (r) { c = !c; } while (c) { l[j] = l[j - g]; j -= g; if (j < g) { break; } c = (l[j - g] > t); if (r) { c = !c; } } } l[j] = t; } } } int[] snortintindi (int[] l) { int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int n = l.length; int[] o = new int[n]; for (int i = 0; i < n; i++) { o[i] = i; } foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int q = o[i]; int j = i; while (j >= g && (l[j - g] > t)) { l[j] = l[j - g]; o[j] = o[j - g]; j -= g; } l[j] = t; o[j] = q; } } return o; } public class MyDat : Object { public int cc; public int[] dat; public string printseq (int[] s, int f) { string[] ii = {"","{","[","[","| ","",""}; string[] dd = {";",","," ",","," | "," ","\n"}; string[] oo = {"","}","]","]"," |","",""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%d".printf(s[i]) + dd[f]); } o = (o + "%d".printf(s[s.length - 1]) + oo[f]); return o; } public string printrow (int p, int f) { int64 tts = GLib.get_monotonic_time(); string[] ii = {"","{","[","[","| ","",""}; string[] dd = {";",","," ",","," | "," ","\n"}; string[] oo = {"","}","]","]"," |","",""}; string o = ii[f]; int a = p * cc; if ((p < 0) || (p > cc)) { int r = (int) (dat.length / cc) - 1; a = (r * cc); } int b = a + cc; for (int i = a; i < (b - 1); i++) { o = (o + "%d".printf(dat[i]) + dd[f]); } o = (o + "%d".printf(dat[b - 1]) + oo[f]); int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t printrow took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t printrow took %.2f microseconds\n",((double) (tte - tts))); } return o; } public int[] copyrow (int p) { int64 tts = GLib.get_monotonic_time(); int[] o = new int[cc]; int a = (p * cc); if ((p < 0) || (p > cc)) { int r = (int) (dat.length / cc) - 1; a = (r * cc); } int b = a + cc; for (int i = a; i < b; i++) { o[i-a] = dat[i]; } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t copyrow took %.2f microseconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t copyrow took %.2f microseconds\n",((double) (tte - tts))); } return o; } public int getcell (int r, int c) { return ((int) ((r * cc) + c)); } public int copycell (int r, int c) { return dat[((int) ((r * cc) + c))]; } public int[] getrow (int p) { //int64 tts = GLib.get_monotonic_time(); int[] o = new int[cc]; int a = (p * cc); if ((p < 0) || (p > cc)) { int r = (int) (dat.length / cc) - 1; a = (r * cc); } int b = a + cc; for (int i = a; i < b; i++) { o[i-a] = i; } //int64 tte = GLib.get_monotonic_time(); //print("\tgetcol took %.2f microseconds\n",((double) (tte - tts))); return o; } public void putrow (int p, int[] c) { //int64 tts = GLib.get_monotonic_time(); int a = (p * cc); if ((p < 0) || (p > cc)) { int r = (int) (dat.length / cc) - 1; a = (r * cc); } int b = a + cc; for (int i = a; i < b; i++) { dat[i] = c[i-a]; } //int64 tte = GLib.get_monotonic_time(); //print("\tputcol took %.2f microseconds\n",((double) (tte - tts))); } public void appendrow (int[] c) { int p = dat.length; dat.resize(dat.length + cc); for (int i = p; i < dat.length; i++) { dat[i] = c[i - p]; } } public void insertrow (int p, int[] c) { int64 tts = GLib.get_monotonic_time(); int a = p * cc; if ((p < 0) || (p > cc)) { int r = (int) (dat.length / cc); a = (r * cc); } int n = cc - 1; dat.resize(dat.length + cc); for (int i = (dat.length - 1); i >= a; i--) { if (i >= (a + cc)) { dat[i] = dat[i - cc]; } else { dat[i] = c[n]; n -= 1; } } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t insertrow took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t insertrow took %.2f microseconds\n",((double) (tte - tts))); } } public void remrow (int p) { int64 tts = GLib.get_monotonic_time(); int a = (p * cc); for (int i = a; i < dat.length; i++) { if ((i + cc) < dat.length) { dat[i] = dat[i + cc]; } } dat.resize(dat.length - cc); int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t remrow took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t remrow took %.2f microseconds\n",((double) (tte - tts))); } } public void remrows (int[] p) { int64 tts = GLib.get_monotonic_time(); int[] sp = p; snortint(sp,true); int n = 0; int a = (sp[sp.length - 1] * cc); int j = cc * p.length; int f = 0; for (int i = a; i < (dat.length - j); i++) { int b = (i + f) / cc; while (b in sp) { f += cc; b += 1; sp.resize(sp.length - 1); } dat[i] = dat[i + f]; } dat.resize(dat.length - j); int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t remrow took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t remrow took %.2f microseconds\n",((double) (tte - tts))); } } public void swaprows (int f, int t) { int tmp = 0; int n = 0; for (int i = (t * cc); i < ((t * cc) + cc); i++) { tmp = dat[i]; dat[i] = dat[((f * cc) + n)]; dat[((f * cc) + n)] = tmp; n += 1; } } public void duperow (int f, int t) { int tmp = 0; int n = 0; for (int i = (t * cc); i < ((t * cc) + cc); i++) { tmp = dat[i]; dat[i] = dat[((f * cc) + n)]; //dat[((f * cc) + n)] = tmp; n += 1; } } public int[] getcol (int p) { int j = (int) (dat.length / cc); int[] o = new int[j]; int n = 0; for (int i = 0; i < dat.length; i++) { if (((i - p) % cc) == 0) { o[n] = i; n += 1; } } return o; } public int[] copycol (int p) { int j = (int) (dat.length / cc); int[] o = new int[j]; int n = 0; for (int i = 0; i < dat.length; i++) { if (((i - p) % cc) == 0) { o[n] = dat[i]; n += 1; } } return o; } public void sortrowsbycol (int r, bool v) { int64 tts = GLib.get_monotonic_time(); int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int[] l = copycol(r); int n = l.length; foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int j = i; bool cond = true; while (cond) { if (j < g) { break; } cond = (l[j - g] > t); if (v) { cond = !cond; } if (cond) { l[j] = l[j - g]; swaprows((j-g),j); j -= g; } } l[j] = t; } } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t sortrowsbycol took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t sortrowsbycol took %.2f microseconds\n",((double) (tte - tts))); } } public int[] getduperowsbycol (int c) { // // +---+---+---+---+---+---+---+---+ // | 4 | 1 | 1 | 1 | 3 | 1 | 7 | 7 | DATA // +---+---+---+---+---+---+---+---+ // | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | POSITION // +---+---+---+---+---+---+---+---+ // | 0 |[2]| 0 | 0 | 0 | 0 |[1]| 0 | DUPE RANGE 0~N // +---+---+---+---+---+---+---+---+ // | | 1 | | | 4 | DUPE FROM POSITION // | | 0 1 [2]| | | 0 [1]| DUPE INDEX // +---+---+---+---+---+---+-------+ // = {0,2,0,0,0,0,1,0} // int64 tts = GLib.get_monotonic_time(); int[] l = getcol(c); int[] o = new int[l.length]; // duped rows for (int i = 0; i < (l.length - 1); i++) { int n = 0; while ((dat[l[i]] - dat[l[i+1+n]]) == 0) { o[i] += 1; n += 1; if ((i + n) >= (l.length - 1)) { break; } } i += n; } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t getdupesbycol took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t getdupesbycol took %.2f microseconds\n",((double) (tte - tts))); } return o; } public void putcol (int p, int[] c) { //int64 tts = GLib.get_monotonic_time(); int n = 0; for (int i = 0; i < dat.length; i++) { if (((i - p) % cc) == 0) { dat[i] = c[n]; n += 1; } } //int64 tte = GLib.get_monotonic_time(); //print("\tputcol took %.2f microseconds\n",((double) (tte - tts))); } public void sortrowsbycolifdupesinothercol (int c, int dc, bool v) { int64 tts = GLib.get_monotonic_time(); int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; int[] d = getduperowsbycol(dc); for (int x = 0; x < d.length; x++) { if (d[x] > 0) { int[] l = {}; int[] lx = {}; for (int a = x; a <= (x + d[x]); a++) { int cel = getcell(a,c); l += dat[cel]; lx += a; } int n = l.length; if (n > 0) { print("\t cell data to sort: %s\n",printseq(l,1)); print("\t rows to reorder: %s\n\n",printseq(lx,1)); foreach (int g in gaps) { for (int i = 0; i < n; i++) { int t = l[i]; int j = i; bool cond = true; while (cond) { if (j < g) { break; } cond = (l[j - g] > t); if (v) { cond = !cond; } if (cond) { l[j] = l[j - g]; swaprows(lx[(j-g)],lx[j]); j -= g; } } l[j] = t; } } } } } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 100000) { print("\t sortrowsbycolifdupesinothercol took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t sortrowsbycolifdupesinothercol took %.2f microseconds\n",((double) (tte - tts))); } } public void filtercols (int k, int op, bool n) { int64 tts = GLib.get_monotonic_time(); // op 0 = starts_with // op 1 = ends_with // op 2 = contains // op 3 = equals // op 4 = greater than // op 5 = less than // n = not int[] ts = {}; for (int i = 0; i < dat.length; i++) { int ci = (i % cc); if (op == 0) { unichar aa = "%d".printf(dat[i]).get_char(0); unichar bb = "%d".printf(k).get_char(0); bool c = (aa == bb); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 1) { //string aa = "%d".printf(dat[i]); //aa = aa.substring(aa.length - 1); //string bb = "%d".printf(k); //bb = bb.substring(bb.length - 1); bool c = ((dat[i] % 10) == (k % 10)); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 2) { bool c = ("%d".printf(dat[i]).contains("%d".printf(k))); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 3) { bool c = (dat[i] == k); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 4) { bool c = (dat[i] > k); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } if (op == 5) { bool c = (dat[i] < k); if (n) { c = !c; } if (c) { if ((ci in ts) == false) { ts += ci; } } } } int[] cids = new int[cc]; for (int i = 0; i < cc; i++) { cids[i] = i; } int[] rt = {}; for (int i = 0; i < cids.length; i++) { if ((cids[i] in ts) == false) { rt += cids[i]; } } if (rt.length > 0) { remrows(rt); } int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t filtercols took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t filtercols took %.2f microseconds\n",((double) (tte - tts))); } } public string printdat (int ind, bool h) { print("cc=%d\n",cc); //int64 tts = GLib.get_monotonic_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string dlm = " | "; int[] cw = new int[cc]; int pad = "%d".printf(cc).char_count(); for (int i = 0; i < cw.length; i++) { cw[i] = pad + 3; } string o = ""; string ln = ""; // get counts 1st for (int i = 0; i < dat.length; i++) { int r = (i % cc); string d = "%d".printf(dat[i]); int cco = 0; if (d != "") { cco = d.char_count() + 3; } cw[r] = int.max(cw[r],cco); } // print line for (int i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s+-".printf(tabs); } ln = "%s%s%*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-+\n".printf(ln); } } o = "%s%s".printf(o,ln); if (h) { // print header for (int i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } string hh = "%0*d".printf(pad,i); int cco = pad; o = "%s%s%*s%s".printf(o,dlm,(cw[i]-cco)," ",hh); if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); } // make rows for (int i = 0; i < dat.length; i++) { int r = (i % cc); int cco = "%d".printf(dat[i]).char_count(); dlm = " | "; if (r == 0) { dlm = "%s| ".printf(tabs); } o = "%s%s%*s%d".printf(o,dlm,(cw[r]-cco)," ",dat[i]); if (r == (cc - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); //int64 tte = GLib.get_monotonic_time(); //print("printdat took %.2f microseconds\n",((double) (tte - tts))); return o; } public void readstream (string p) { int64 tts = GLib.get_monotonic_time(); StringBuilder xx = new StringBuilder(""); string ff = Path.build_filename ("./", p); GLib.File fff = File.new_for_path(ff); if (fff.query_exists() == true) { try { uint8[] c; string e; fff.load_contents (null, out c, out e); xx.append((string) c); } catch (Error e) { print ("\tfailed to read %s: %s\n", fff.get_path(), e.message); } } else { print("file not found %s\n",ff); } int oo = xx.str.index_of(";"); string h = xx.str.substring(0,oo); cc = int.parse(h); oo = xx.str.index_of("\n"); string tt = xx.str.substring(oo+1); xx.erase(0,-1); xx.append(tt); string[] tdat = xx.str.split("\n"); xx.erase(0,-1); dat.resize(tdat.length); for (int s = 0; s < tdat.length; s++) { dat[s] = int.parse(tdat[s]); } tdat = null; int64 tte = GLib.get_monotonic_time(); print("readstream took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } public void setdat (int[] d, int c) { dat.resize(d.length); cc = c; for (int i = 0; i < d.length; i++) { dat[i] = d[i]; } } public MyDat (int l) { cc = l; dat = {}; } } void main() { MyDat m = new MyDat(3); m.setdat({0,0,0,1,1,1,2,2,2,3,3,3,4,4,4,5,5,5,6,6,6},3); print("\ndat:\n"); print("%s\n",m.printdat(1,false)); print("\nremrows({5,0,1}):\n"); int[] nc = {5,0,1}; m.remrows(nc); print("%s\n",m.printdat(1,true)); print("\ninsertrow(1,{0,0,0}):\n"); nc = {0,0,0}; m.insertrow(1,nc); print("%s\n",m.printdat(1,true)); print("\ninsertrow(-1,{7,7,7}) (append):\n"); nc = {7,7,7}; m.insertrow(-1,nc); print("%s\n",m.printdat(1,true)); print("\nappendrow({8,8,8}):\n"); nc = {8,8,8}; m.appendrow(nc); print("%s\n",m.printdat(1,true)); print("\nputrow(0,{9,9,9}):\n"); nc = {9,9,9}; m.putrow(0,nc); print("%s\n",m.printdat(1,true)); print("\ngetrow(0):\n"); nc = m.getrow(0); print("\t= {"); for (int i = 0; i < (nc.length - 1); i++) { print("%d,",nc[i]); } print("%d}\n",nc[nc.length -1]); print("\ngetrow(6):\n"); nc = m.getrow(6); print("\t= {"); for (int i = 0; i < (nc.length - 1); i++) { print("%d,",nc[i]); } print("%d}\n",nc[nc.length -1]); print("\ngetrow(-1):\n"); nc = m.getrow(-1); print("\t= {"); for (int i = 0; i < (nc.length - 1); i++) { print("%d,",nc[i]); } print("%d}\n",nc[nc.length -1]); print("\ncopyrow(0):\n"); nc = m.copyrow(0); print("\t= {"); for (int i = 0; i < (nc.length - 1); i++) { print("%d,",nc[i]); } print("%d}\n",nc[nc.length -1]); print("\ncopyrow(6):\n"); nc = m.copyrow(6); print("\t= {"); for (int i = 0; i < (nc.length - 1); i++) { print("%d,",nc[i]); } print("%d}\n",nc[nc.length -1]); print("\ncopyrow(-1):\n"); nc = m.copyrow(-1); print("\t= {"); for (int i = 0; i < (nc.length - 1); i++) { print("%d,",nc[i]); } print("%d}\n",nc[nc.length -1]); // 0 = csc // 1 = vala // 2 = rebol // 3 = python // 4 = org // 5 = words // 6 = lines print("\nprintrow(-1,0):\n"); print("\t%s\n",m.printrow(-1,0)); print("\nprintrow(-1,1):\n"); print("\t%s\n",m.printrow(-1,1)); print("\nprintrow(-1,2):\n"); print("\t%s\n",m.printrow(-1,2)); print("\nprintrow(-1,3):\n"); print("\t%s\n",m.printrow(-1,3)); print("\nprintrow(-1,4):\n"); print("\t%s\n",m.printrow(-1,4)); print("\nprintrow(-1,5):\n"); print("\t%s\n",m.printrow(-1,5)); print("\nprintrow(-1,6):\n"); print("%s\n",m.printrow(-1,6)); print("\nswaprows(0,6):\n"); m.swaprows(0,6); print("%s\n",m.printdat(1,true)); print("\nduperow(5,6):\n"); m.duperow(5,6); print("%s\n",m.printdat(1,true)); print("\nsortrowsbycol(1,false):\n"); m.sortrowsbycol(1,false); print("%s\n",m.printdat(1,true)); print("\nsortrowsbycol(1,true) (reverse):\n"); m.sortrowsbycol(1,true); print("%s\n",m.printdat(1,true)); print("\nduperow(6,5):\n"); m.duperow(6,5); print("\nduperow(6,4):\n"); m.duperow(6,4); print("%s\n",m.printdat(1,true)); print("\ngetduperowsbycol(1):\n"); nc = m.getduperowsbycol(1); print("\t= {"); for (int i = 0; i < (nc.length - 1); i++) { print("%d,",nc[i]); } print("%d}\n",nc[nc.length -1]); print("\nputcol(0,{}):\n"); nc = {3,8,1,0,5,2,4}; m.putcol(0,nc); print("%s\n",m.printdat(1,true)); print("\nsortrowsbycolifdupesinothercol(0,1,false):\n"); m.sortrowsbycolifdupesinothercol(0,2,false); print("%s\n",m.printdat(1,true)); print("\ngetcell(0,2):\n"); print("%d\n",m.getcell(0,2)); print("\ncopycell(0,2):\n"); print("%d\n",m.copycell(0,2)); }
#+RESULTS:
Compilation succeeded - 2 warning(s) dat: cc=3 +------+------+------+ | 0 | 0 | 0 | | 1 | 1 | 1 | | 2 | 2 | 2 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 5 | 5 | 5 | | 6 | 6 | 6 | +------+------+------+ remrows({5,0,1}): remrow took 2.00 microseconds cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 2 | 2 | 2 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | +------+------+------+ insertrow(1,{0,0,0}): insertrow took 1.00 microseconds cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 2 | 2 | 2 | | 0 | 0 | 0 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | +------+------+------+ insertrow(-1,{7,7,7}) (append): insertrow took 0.00 microseconds cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 2 | 2 | 2 | | 0 | 0 | 0 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | | 7 | 7 | 7 | +------+------+------+ appendrow({8,8,8}): cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 2 | 2 | 2 | | 0 | 0 | 0 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | | 7 | 7 | 7 | | 8 | 8 | 8 | +------+------+------+ putrow(0,{9,9,9}): cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 9 | 9 | 9 | | 0 | 0 | 0 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | | 7 | 7 | 7 | | 8 | 8 | 8 | +------+------+------+ getrow(0): = {0,1,2} getrow(6): = {18,19,20} getrow(-1): = {18,19,20} copyrow(0): copyrow took 0.00 microseconds = {9,9,9} copyrow(6): copyrow took 1.00 microseconds = {8,8,8} copyrow(-1): copyrow took 1.00 microseconds = {8,8,8} printrow(-1,0): printrow took 4.00 microseconds 8;8;8 printrow(-1,1): printrow took 2.00 microseconds {8,8,8} printrow(-1,2): printrow took 2.00 microseconds [8 8 8] printrow(-1,3): printrow took 2.00 microseconds [8,8,8] printrow(-1,4): printrow took 2.00 microseconds | 8 | 8 | 8 | printrow(-1,5): printrow took 2.00 microseconds 8 8 8 printrow(-1,6): printrow took 2.00 microseconds 8 8 8 swaprows(0,6): cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 8 | 8 | 8 | | 0 | 0 | 0 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | | 7 | 7 | 7 | | 9 | 9 | 9 | +------+------+------+ duperow(5,6): cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 8 | 8 | 8 | | 0 | 0 | 0 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | | 7 | 7 | 7 | | 7 | 7 | 7 | +------+------+------+ sortrowsbycol(1,false): sortrowsbycol took 3.00 microseconds cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 0 | 0 | 0 | | 3 | 3 | 3 | | 4 | 4 | 4 | | 6 | 6 | 6 | | 7 | 7 | 7 | | 7 | 7 | 7 | | 8 | 8 | 8 | +------+------+------+ sortrowsbycol(1,true) (reverse): sortrowsbycol took 2.00 microseconds cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 8 | 8 | 8 | | 7 | 7 | 7 | | 7 | 7 | 7 | | 6 | 6 | 6 | | 4 | 4 | 4 | | 3 | 3 | 3 | | 0 | 0 | 0 | +------+------+------+ duperow(6,5): duperow(6,4): cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 8 | 8 | 8 | | 7 | 7 | 7 | | 7 | 7 | 7 | | 6 | 6 | 6 | | 0 | 0 | 0 | | 0 | 0 | 0 | | 0 | 0 | 0 | +------+------+------+ getduperowsbycol(1): getdupesbycol took 1.00 microseconds = {0,1,0,0,2,0,0} putcol(0,{}): cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 3 | 8 | 8 | | 8 | 7 | 7 | | 1 | 7 | 7 | | 0 | 6 | 6 | | 5 | 0 | 0 | | 2 | 0 | 0 | | 4 | 0 | 0 | +------+------+------+ sortrowsbycolifdupesinothercol(0,1,false): getdupesbycol took 0.00 microseconds cell data to sort: {8,1} rows to reorder: {1,2} cell data to sort: {5,2,4} rows to reorder: {4,5,6} sortrowsbycolifdupesinothercol took 32.00 microseconds cc=3 +------+------+------+ | 0 | 1 | 2 | +------+------+------+ | 3 | 8 | 8 | | 1 | 7 | 7 | | 8 | 7 | 7 | | 0 | 6 | 6 | | 2 | 0 | 0 | | 4 | 0 | 0 | | 5 | 0 | 0 | +------+------+------+ getcell(0,2): 2 copycell(0,2): 8
RUNNINGTOTAL:vala:
bool isanum (string s) { StringBuilder cls = new StringBuilder(""); cls.append(s); unichar[] np = {'€','$','-','+','.',',','\''}; for (int x = 0; x < cls.str.char_count(); x++) { int i = cls.str.index_of_nth_char(x); unichar u = cls.str.get_char(i); if (u.isdigit() == false) { if (u in np) { continue; } return false; } } return true; } string printunrolledarray (int ind, string[] dat, int dsz, bool h, bool rnm, bool cnm, int m) { string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string spc = " "; bool domask = (m > -1); string dlm = " | "; int rcols = dsz; int rowcount = dat.length / dsz; if (rnm) { rcols = dsz+1; } int chc = 0; if (cnm) { chc = "%d".printf((dat.length - 1)).char_count(); } int[] cw = new int[rcols]; for (int i = 0; i < cw.length; i++) { cw[i] = 0; } if (rnm) { cw[0] = "%d".printf((dat.length/dsz)).char_count(); } if (h) { for (int i = 1; i < cw.length; i++) { string hh = "col %u".printf(i); cw[i] = hh.char_count(); } } string o = ""; string ln = ""; // get counts 1st for (int i = 0; i < dat.length; i++) { int c = (i % dsz); if (rnm) { c = c + 1; } int cc = 0; if (dat[i] != null) { cc = dat[i].char_count() + chc + 3; if (cc > 0) { int nwlat = dat[i].index_of("\n"); if (nwlat != -1) { cc = (nwlat + 3 + chc + 3); } } } cw[c] = int.max(cw[c],cc); } // print header if (h) { for (int i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } if (i == 0) { string hh = "#"; o = "%s%s%-*s".printf(o,dlm,cw[i],hh); } else { string hh = "col %d".printf(i-1); o = "%s%s%-*s".printf(o,dlm,cw[i],hh); } if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } // print line for (int i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s|-".printf(tabs); } ln = "%s%s%-*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-|\n".printf(ln); } } o = "%s%s".printf(o,ln); } int n = 0; int rcc = dsz + 1; for (int r = 0; r < rowcount; r++) { for (int c = 0; c < rcc; c++) { string datpart = ""; if (c == 0) { dlm = "%s| ".printf(tabs); o = "%s%s%0*d".printf(o,dlm,cw[0],r); } else { if (c == 1) { dlm = "%s| ".printf(tabs); } int j = (r * dsz) + (c - 1); int cc = dat[j].char_count(); if (dat[j] != null) { if (cc > 0) { datpart = dat[j]; int nwlat = dat[j].index_of("\n"); if (nwlat != -1) { //trunc = true; datpart = (dat[j].substring(0,nwlat) + "..."); cc = (nwlat + 3); } } else { cc = int.max(cc,1); } } dlm = " | "; string clen = ""; if (cnm) { clen = " (%*d)".printf(chc,j); } int padlen = (cw[c] - cc) - clen.char_count(); padlen = int.max(padlen,0); datpart = "%s%s%.*s".printf(datpart,clen,padlen,spc); if (domask) { bool ismasked = (((j % dsz) + m) == 0); if (ismasked) { n += 2; datpart = "\x1b[48;5;5m%s\x1b[0m".printf(datpart); } } o = "%s%s%s".printf(o,dlm,datpart); if (c == dsz) { dlm = " |\n"; o = "%s%s".printf(o,dlm); } } } } cw = null; return o; } void main () { GLib.Intl.setlocale(ALL,""); int dsz = 5; string[] dat = { "one","AA","3.0","BB","", "two","CC","$200'000","DD","", "three","EE","$-1000.00","FF","", "four","GG","4.3","HH","", "five","II","1..","JJ","", "six","KK","6.4","LL","", "six","MM","-€2000,00","NN","" }; int oc = 2; int ic = 4; double bal = 0.0; int ofs = ic-oc; for (int i = oc; i < dat.length; i += dsz) { double nbal = 0.0; if (isanum(dat[i])) { StringBuilder cls = new StringBuilder(""); cls.append(dat[i]); if (cls.str.index_of_char('€') > -1) { cls.replace(",","."); cls.replace("€",""); } cls.replace(",",""); cls.replace("\'",""); cls.replace("$",""); cls.replace("+",""); print("cleaned %s is %s\n",dat[i],cls.str); nbal = (bal + double.parse(cls.str)); } else { nbal = bal + nbal; } dat[i+ofs] = "%.2f".printf(nbal); bal = nbal; } print("%s\n",printunrolledarray(1, dat, dsz, true, true, false, -1)); }
#+RESULTS:
cleaned 3.0 is 3.0 cleaned $200'000 is 200000 cleaned $-1000.00 is -1000.00 cleaned 4.3 is 4.3 cleaned 1.. is 1.. cleaned 6.4 is 6.4 cleaned -€2000,00 is -2000.00 | # | col 0 | col 1 | col 2 | col 3 | col 4 | |---+-------+-------+-----------+-------+-----------| | 0 | one | AA | 3.0 | BB | 3.00 | | 1 | two | CC | $200'000 | DD | 200003.00 | | 2 | three | EE | $-1000.00 | FF | 199003.00 | | 3 | four | GG | 4.3 | HH | 199007.30 | | 4 | five | II | 1.. | JJ | 199008.30 | | 5 | six | KK | 6.4 | LL | 199014.70 | | 6 | six | MM | -€2000,00 | NN | 197014.70 |
SCRAPE:busted::vala:

Get rendered text of a html/java document.
Uses webkit, which has to display in a ui before it will do anything else.
There might be a workaround, need more testing.

int main() { string u = "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent"; // TODO: research how to check site existence without leaking infos WebKit.Settings wvs = new WebKit.Settings(); // disable unnecessary web-browser features wvs.set_javascript_can_access_clipboard(false); wvs.set_javascript_can_open_windows_automatically(false); wvs.set_hardware_acceleration_policy(NEVER); wvs.set_enable_webgl(false); wvs.set_enable_webaudio(false); wvs.set_enable_page_cache(false); wvs.set_enable_offline_web_application_cache(false); wvs.set_enable_media_stream(false); wvs.set_enable_html5_local_storage(false); wvs.set_enable_html5_database(false); wvs.set_user_agent_with_application_details("Mozilla","5.0"); wvs.set_allow_modal_dialogs(false); wvs.set_allow_file_access_from_file_urls(false); print("webview user agent is: %s\n",wvs.get_user_agent()); print("loading url: %s ...\n",u); Gtk.Application myapp = new Gtk.Application("com.mytest.webkittest",GLib.ApplicationFlags.DEFAULT_FLAGS); myapp.activate.connect (() => { Gtk.Box listbox = new Gtk.Box(VERTICAL,0); Gtk.Box controlbox = new Gtk.Box(HORIZONTAL,5); Gtk.Button jsbutton = new Gtk.Button.with_label("run"); Gtk.Entry addy = new Gtk.Entry(); Gtk.Button gobutton = new Gtk.Button.with_label("go"); Gtk.ApplicationWindow mywin = new Gtk.ApplicationWindow(myapp); Gtk.ScrolledWindow wvsv = new Gtk.ScrolledWindow(); WebKit.WebView wv = new WebKit.WebView(); wv.set_settings(wvs); wv.vexpand = true; wv.hexpand = true; wvsv.hexpand = true; addy.hexpand = true; controlbox.hexpand = false; controlbox.append(jsbutton); controlbox.append(addy); controlbox.append(gobutton); wvsv.set_child(wv); listbox.append(wvsv); listbox.append(controlbox); mywin.set_child(listbox); mywin.default_width = 800; mywin.default_height = 800; mywin.present(); StringBuilder tidyurl = new StringBuilder(""); gobutton.clicked.connect(() => { u = addy.text.strip(); tidyurl.erase(0,-1); tidyurl.append(u); tidyurl.replace(".html",""); tidyurl.replace("http://",""); tidyurl.replace("https://",""); tidyurl.replace("www.",""); tidyurl.replace("/","_"); tidyurl.replace(".","_"); tidyurl.replace("?",""); tidyurl.replace("%%",""); tidyurl.replace("=",""); tidyurl.replace("-",""); tidyurl.replace("+",""); tidyurl.replace("*",""); tidyurl.replace("\\",""); tidyurl.replace("&",""); wv.load_uri(u); }); wv.load_changed.connect((lev) => { if (lev == FINISHED) { print("page loaded: %s\n",wv.uri); addy.text = wv.uri; u = addy.text.strip(); tidyurl.erase(0,-1); tidyurl.append(u); tidyurl.replace(".html",""); tidyurl.replace("http://",""); tidyurl.replace("https://",""); tidyurl.replace("www.",""); tidyurl.replace("/","_"); tidyurl.replace(".","_"); tidyurl.replace("?",""); tidyurl.replace("%%",""); tidyurl.replace("=",""); tidyurl.replace("-",""); tidyurl.replace("+",""); tidyurl.replace("*",""); tidyurl.replace("\\",""); tidyurl.replace("&",""); } }); jsbutton.clicked.connect(() => { StringBuilder extracted = new StringBuilder(""); print("evaluating script...\n"); Value h = new Value(typeof(string)); StringBuilder pagetitle = new StringBuilder(""); pagetitle.append(wv.title); pagetitle.replace(" ","_",-1); //wv.evaluate_javascript( // "document.write(document.body.innerText)",-1,null,null,null //); //const links = document.getElementsByTagName('a'); //for (let link of links) { console.log(link.href); } wv.evaluate_javascript( "document.body.innerText",-1,null,null,null,(webv,asyres) => { print("result type is %s\n",asyres.get_type().name()); WebKit.WebView wwebv = (WebKit.WebView) webv; var huh = wwebv.evaluate_javascript.end(asyres); extracted.append(huh.to_string()); //print("result val is %s\n",extracted.str); //GLib.Thread.usleep(5000); if (extracted.str.char_count() > 0) { if (extracted.str.strip().char_count() > 0) { print("OK\n"); print("saving scraped text..."); string l = ("./" + tidyurl.str + "_" + pagetitle.str + ".txt"); File lf = File.new_for_path(l); try { FileOutputStream lfs = lf.replace(null,false,FileCreateFlags.PRIVATE); lfs.write(extracted.data); print("DONE, saved to: %s\n",l); } catch (Error e) { print("error writing to %s:\n\t%s\n",l,e.message); } } else { print("error: extracted text is emty\n"); } } else { print("error: extracted text is empty\n"); } } ); }); //wv.terminate_web_process(); //wv.try_close(); }); return myapp.run(); }
#+RESULTS:
Compilation succeeded - 4 warning(s) webview user agent is: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15 Mozilla/5.0 loading url: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent ... page loaded: https://theconversation.com/au page loaded: https://theconversation.com/in-failing-to-probe-robodebt-australias-anti-corruption-body-fell-at-the-first-hurdle-it-now-has-a-second-chance-236147 evaluating script... result type is GTask OK saving scraped text...DONE, saved to: ./theconversation_com_infailingtoproberobodebtaustraliasanticorruptionbodyfellatthefirsthurdleitnowhasasecondchance236147_In_failing_to_probe_Robodebt,_Australia’s_anti-corruption_body_fell_at_the_first_hurdle._It_now_has_a_second_chance.txt
SETCSS:vala:
// css test // by c.p.brown 2023 using Gtk; int main (string[] args) { Gtk.Application wut = new Gtk.Application ("com.test.test", GLib.ApplicationFlags.DEFAULT_FLAGS); wut.activate.connect(() => { Gtk.ApplicationWindow win = new Gtk.ApplicationWindow(wut); Gtk.Box container = new Gtk.Box(VERTICAL,0); Gtk.Box socket = new Gtk.Box(HORIZONTAL,0); Gtk.ToggleButton btn = new Gtk.ToggleButton(); string csses = """ .rr { background: #FF5555; } .bb { background: #5555FF; } .gg { background: #55ff55; } """; Gtk.CssProvider allcsp = new Gtk.CssProvider(); allcsp.load_from_string(csses); Gdk.Display thisdisplay = Gdk.Display.get_default(); Gtk.StyleContext.add_provider_for_display(thisdisplay, allcsp, Gtk.STYLE_PROVIDER_PRIORITY_USER); container.set_css_classes({"bb"}); socket.set_css_classes({"gg"}); btn.set_css_classes({"rr"}); socket.append(btn); container.append(socket); win.set_child(container); win.default_width = 300; win.default_height = 300; win.present(); }); return wut.run(args); }
SETPROPERTY:vala:
// force nested subclassed gtk widgets to talk to each other // since they won't by default for reasons unknown // by c.p.brown 2024 public class bbb : Gtk.Button { public string bbvar {get;set;} public bbb () { this.clicked.connect(() => { this.set_label(bbvar); }); } } int main() { Gtk.Application wut = new Gtk.Application ("com.test.test", GLib.ApplicationFlags.DEFAULT_FLAGS); wut.activate.connect(() => { Gtk.ApplicationWindow win = new Gtk.ApplicationWindow(wut); bbb o = new bbb(); Gtk.Box container = new Gtk.Box(VERTICAL,0); container.append(o); Object huh = (Object) container.get_first_child(); Value duh = Value(typeof(string)); duh.set_string("ONE"); huh.set_property("bbvar",duh); win.set_child(container); win.default_width = 300; win.default_height = 300; win.present(); }); return wut.run(); }
#+RESULTS:

SHELLSORT:vala:

Another shellsort test.
This is derivative work, from various sources including Wikipedia & Rosetta Code.

void snortint (ref int[] p) { int64 tts = GLib.get_monotonic_time(); if (p.length > 0) { for (int k = 0; k < p.length; k++) { print("pre shellsort %d = %d\n",k,p[k]); } int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; foreach (int g in gaps) { for (int i = 0; i < p.length; i++) { int t = p[i]; int j = i; bool cond = true; while (cond) { if (j < g) { break; } cond = (p[j - g] > t); if (cond) { p[j] = p[j - g]; j -= g; } } p[j] = t; } } print("\n"); for (int k = 0; k < p.length; k++) { print("post shellsort %d = %d\n",k,p[k]); } } print("\n"); int64 tte = GLib.get_monotonic_time(); if ((tte - tts) > 500000) { print("\t shellsort took %.6f seconds\n",((double) (tte - tts) / 1000000.0)); } else { print("\t shellsort took %.2f microseconds\n",((double) (tte - tts))); } } void main() { int[] s = {5,4,1,9,8,6,0,3,7,2}; snortint(ref s); }
#+RESULTS:
pre shellsort 0 = 5 pre shellsort 1 = 4 pre shellsort 2 = 1 pre shellsort 3 = 9 pre shellsort 4 = 8 pre shellsort 5 = 6 pre shellsort 6 = 0 pre shellsort 7 = 3 pre shellsort 8 = 7 pre shellsort 9 = 2 post shellsort 0 = 0 post shellsort 1 = 1 post shellsort 2 = 2 post shellsort 3 = 3 post shellsort 4 = 4 post shellsort 5 = 5 post shellsort 6 = 6 post shellsort 7 = 7 post shellsort 8 = 8 post shellsort 9 = 9 shellsort took 114.00 microseconds
SORT STRING NUMBERS:vala:

string printstringarray (string[] s, int f) { // 0 1 2 3 4 5 6 7 8 string[] ii = {"", "\"", "{", "[", "[", "| ", "", "", ""}; string[] dd = {";", ",", ",", " ", ",", " | ", " ", "\n", ",\n"}; string[] oo = {"", "\"", "}", "]", "]", " |", "", "", ""}; string o = ii[f]; for (int i = 0; i < (s.length - 1); i++) { o = (o + "%s".printf(s[i]) + dd[f]); } o = (o + "%s".printf(s[s.length - 1]) + oo[f]); return o; } string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } int[] numchars; int[] prenums; int strintcmp (string a, string b) { int ach = a.char_count(); int bch = b.char_count(); if ((a.length > 1) && (b.length > 1)) { int ca = ((int) a.get_char(a.index_of_nth_char(0))); int cb = ((int) b.get_char(b.index_of_nth_char(0))); int nn = 0; int na = 1; while (ca in prenums) { if (na >= ach) { nn = 1; break; } ca = ((int) a.get_char(a.index_of_nth_char(na))); na += 1; } //print("last prechar in \"%s\" was at char %d of %d\n",a,(na - 1),bch); int nb = 1; while (cb in prenums) { if (nb >= bch) { nn = 1; break; } cb = ((int) b.get_char(b.index_of_nth_char(nb))); nb += 1; } int ea = na; int eb = nb; if (nn == 0) { na = na - 1; nb = nb - 1; //print("number part of \"%s\" is %s\n",a,a.substring(na)); //print("number part of \"%s\" is %s\n",b,b.substring(nb)); int cea = ((int) a.get_char(a.index_of_nth_char(na))); int ceb = ((int) b.get_char(b.index_of_nth_char(nb))); while (cea in numchars) { if (ea >= a.length) { break; } cea = ((int) a.get_char(a.index_of_nth_char(ea))); ea += 1; } while (ceb in numchars) { if (eb >= b.length) { break; } ceb = ((int) b.get_char(b.index_of_nth_char(eb))); eb += 1; } //ea = ea - 1; //eb = eb - 1; print("number part of \"%s\" is %s\n",a,a.substring(na,ea)); print("number part of \"%s\" is %s\n",b,b.substring(nb,eb)); } if ((nn == 0) && (ca in numchars) && (cb in numchars)) { StringBuilder sba = new StringBuilder(""); StringBuilder sbb = new StringBuilder(""); if ((ea > na) && (ea < (a.length - 1))) { sba.append(a.substring(na,(ea-na))); } else { sba.append(a.substring(na)); } if ((eb > nb) && (eb < (b.length - 1))) { sbb.append(b.substring(nb,(eb-nb))); } else { sbb.append(b.substring(nb)); } //sba.replace("\"","",1); //sba.replace("$","",1); //sba.replace("/","",2); // omit the following for fucky EU numbers where they use commas as decimal-points //sba.replace(",","",0); //sba.replace("\'","",0); //sbb.replace("\"","",1); //sbb.replace("$","",1); //sbb.replace("/","",2); //sbb.replace(",","",0); //sbb.replace("\'","",0); if ((sba.str.length > 0) && (sbb.str.length > 0)) { //print("%s - %s...\n",sba.str,sbb.str); int dd = ((int) (double.parse(sba.str) - double.parse(sbb.str))); if ( (dd == 0) && ((ea > na) && (ea < (a.length - 1))) && ((eb > nb) && (eb < (b.length - 1))) ) { return strcmp(a.substring(ea),b.substring(eb)); } return dd; } } } return strcmp(a,b); } void rawquicksortonedeeold (ref string[] dat, int s, int e) { int itr = 0; if ((e - s) > 0) { int p = s; //(s + ((e - s) / 2)); int i = s; int j = e; while(i <= j) { while (strcmp(dat[i],dat[p]) < 0) { i += 1; } while (strcmp(dat[j],dat[p]) > 0) { j -= 1; } if (i <= j) { string t = dat[i]; dat[i] = dat[j]; dat[j] = t; i += 1; j -= 1; } } if (j < i) { rawquicksortonedeeold(ref dat,s,j); rawquicksortonedeeold(ref dat,i,e); } } } void rawquicksortonedee (ref string[] dat, int s, int e) { if ((e - s) > 0) { int p = s; //(s + ((e - s) / 2)); int i = s; int j = e; while(i <= j) { while (strintcmp(dat[i],dat[p]) < 0) { i += 1; } while (strintcmp(dat[j],dat[p]) > 0) { j -= 1; } if (i <= j) { string t = dat[i]; dat[i] = dat[j]; dat[j] = t; i += 1; j -= 1; } } if (j < i) { rawquicksortonedee(ref dat,s,j); rawquicksortonedee(ref dat,i,e); } } } void main() { GLib.Intl.setlocale(ALL,""); // {'-','+','0','1','2','3','4','5','6','7','8','9','.'}; numchars = {43,45,48,49,50,51,52,53,54,55,56,57,46}; // TAB SPC " $ ' prenums = {9, 32, 34, 36, 39}; string[] s = {"0","20.2","10E","$120","220_C","\"$12,000\"","3"," \"$ 90\'000\"","80","320","3A","118","126","44"}; int64 tts = GLib.get_monotonic_time(); rawquicksortonedee(ref s,0,(s.length - 1)); int64 tte = GLib.get_monotonic_time(); print("\nquicksort using strintcmp took %s\n".printf(printusecs(tte-tts))); print("sorted string using strintcmp:\n%s\n",printstringarray(s,2)); s = {"0","20.2","10E","$120","220_C","\"$12,000\"","3"," \"$ 90\'000\"","80","320","3A","118","126","44"}; int64 tnts = GLib.get_monotonic_time(); rawquicksortonedeeold(ref s,0,(s.length - 1)); int64 tnte = GLib.get_monotonic_time(); print("\nquicksort using strcmp took %s\n".printf(printusecs(tnte-tnts))); print("sorted string using strcmp:\n%s\n\n",printstringarray(s,2)); }
#+RESULTS:
Compilation succeeded - 1 warning(s) number part of "20.2" is 20.2 number part of " "$ 90'000"" is 90'000" number part of "10E" is 10E number part of " "$ 90'000"" is 90'000" number part of "$120" is (null) number part of " "$ 90'000"" is 90'000" number part of ""$12,000"" is 12,00 number part of " "$ 90'000"" is 90'000" number part of "220_C" is 220_ number part of " "$ 90'000"" is 90'000" number part of "220_C" is 220_ number part of " "$ 90'000"" is 90'000" number part of ""$12,000"" is 12,00 number part of " "$ 90'000"" is 90'000" number part of " "$ 90'000"" is 90'000" number part of " "$ 90'000"" is 90'000" number part of ""$12,000"" is 12,00 number part of " "$ 90'000"" is 90'000" number part of "20.2" is 20.2 number part of ""$12,000"" is 12,00 number part of "10E" is 10E number part of ""$12,000"" is 12,00 number part of ""$12,000"" is 12,00 number part of ""$12,000"" is 12,00 number part of "10E" is 10E number part of ""$12,000"" is 12,00 number part of "20.2" is 20.2 number part of "20.2" is 20.2 number part of " "$ 90'000"" is 90'000" number part of "20.2" is 20.2 number part of "20.2" is 20.2 number part of "20.2" is 20.2 number part of "220_C" is 220_ number part of "220_C" is 220_ number part of "44" is 44 number part of "220_C" is 220_ number part of "$120" is (null) number part of "44" is 44 number part of "126" is 126 number part of "44" is 44 number part of "118" is 118 number part of "44" is 44 number part of "3A" is 3A number part of "44" is 44 number part of "80" is 80 number part of "44" is 44 number part of "320" is 320 number part of "44" is 44 number part of "80" is 80 number part of "44" is 44 number part of "44" is 44 number part of "44" is 44 number part of "3A" is 3A number part of "3A" is 3A number part of "44" is 44 number part of "3A" is 3A number part of "3A" is 3A number part of "3A" is 3A number part of "44" is 44 number part of "3A" is 3A number part of "3A" is 3A number part of "3A" is 3A number part of "80" is 80 number part of "80" is 80 number part of "220_C" is 220_ number part of "80" is 80 number part of "126" is 126 number part of "80" is 80 number part of "118" is 118 number part of "80" is 80 number part of "$120" is (null) number part of "80" is 80 number part of "320" is 320 number part of "80" is 80 number part of "80" is 80 number part of "80" is 80 number part of "320" is 320 number part of "320" is 320 number part of "220_C" is 220_ number part of "320" is 320 number part of "$120" is (null) number part of "220_C" is 220_ number part of "118" is 118 number part of "220_C" is 220_ number part of "126" is 126 number part of "220_C" is 220_ number part of "320" is 320 number part of "220_C" is 220_ number part of "126" is 126 number part of "220_C" is 220_ number part of "220_C" is 220_ number part of "220_C" is 220_ number part of "126" is 126 number part of "220_C" is 220_ number part of "$120" is (null) number part of "126" is 126 number part of "118" is 118 number part of "126" is 126 number part of "220_C" is 220_ number part of "126" is 126 number part of "118" is 118 number part of "126" is 126 number part of "126" is 126 number part of "126" is 126 number part of "118" is 118 number part of "126" is 126 number part of "$120" is (null) number part of "118" is 118 number part of "$120" is (null) number part of "118" is 118 number part of "118" is 118 number part of "118" is 118 number part of "$120" is (null) number part of "$120" is (null) number part of "126" is 126 number part of "$120" is (null) number part of "$120" is (null) number part of "$120" is (null) quicksort using strintcmp took 315 μs sorted string using strintcmp: {10E,"$12,000",20.2, "$ 90'000",0,3,3A,44,80,118,$120,126,220_C,320} quicksort using strcmp took 3 μs sorted string using strcmp: { "$ 90'000","$12,000",$120,0,10E,118,126,20.2,220_C,3,320,3A,44,80}
SPLIT:vala:
NOTE: split may return empty strings when the delimiter is at either end of the string
string ";1;2;3;4;5; split ^|^|^|^|^|^ |;|;|;|;|;|;| |?|1|2|3|4|5|?| partitions |1|2|3|4|5|6|7|
void main(string[] args) { string s = "| 1 | 2 | 3 | 4 | 5 |"; string[] p = s.split("|"); print("last item : %s\n",p[p.length - 1]); print("p.length = %d\n",p.length); }
#+RESULTS:
last item : p.length = 7
STRINGBUILDER:vala:

StringBuilder tests

Notes:
  • erase(x,y) is from-and-including x, to-and-including x+y
  • erase(x,-1) sometimes fails in real-world use

void main() { GLib.Intl.setlocale(ALL,""); string s = """零1“23456789”"""; StringBuilder sb = new StringBuilder(s); print(sb.str); string n = ("“"); int ii = sb.str.index_of(n); int oo = sb.str.index_of("2"); print("\nindex of string \"“\" is %d\n", ii); print("str[%d] is \'(invalid)\'\n", ii); print("str.get_char(%d) is \"%s\"\n\n", ii, sb.str.get_char(ii).to_string()); print("\nindex of string \"2\" is %d\n", oo); print("str[%d] is \'%s\'\n", oo, sb.str[oo].to_string()); print("str.get_char(%d) is \"%s\"\n\n", oo, sb.str.get_char(oo).to_string()); sb.erase(ii,n.length); print("sb.erase(%d,%d) is %s\n",ii,n.length,sb.str); sb.insert(ii,"\""); print("sb.insert(%d,\") is %s\n",ii,sb.str); int qq = oo + ("\"".length - n.length); print("\nindex of string \"2\" is now %d\n", qq); print("str[%d] is \'%s\'\n", qq, sb.str[qq].to_string()); print("str.get_char(%d) is \"%s\"\n\n", qq, sb.str.get_char(qq).to_string()); ii = sb.str.index_of_char('2'); print("\nindex of char \'2\' is %d\n", ii); int rem = (sb.str.length - ii); print("sb.str.legth is %d\n", sb.str.length); print("%d - %d is %d\n",sb.str.length,ii,rem); sb.erase(ii,rem); print("sb.erase(%d,%d) is %s\n",ii,rem,sb.str); sb.str = """零1"23456789”"""; print("\nstr is now %s\n",sb.str); ii = sb.str.index_of_char('2'); print("\nindex of char \'2\' is now %d\n", ii); sb.erase(ii,-1); print("sb.erase(%d,-1) is %s\n",ii,sb.str); }
#+RESULTS:
零1“23456789” index of string "“" is 4 str[4] is '(invalid)' str.get_char(4) is "“" index of string "2" is 7 str[7] is '2' str.get_char(7) is "2" sb.erase(4,3) is 零123456789” sb.insert(4,") is 零1"23456789” index of string "2" is now 5 str[5] is '2' str.get_char(5) is "2" index of char '2' is 5 sb.str.legth is 16 16 - 5 is 11 sb.erase(5,11) is 零1" str is now 零1"23456789” index of char '2' is now 5 sb.erase(5,-1) is 零1"
STRFTIME:vala:

DateTime
  • has time and timezone
  • can get components as integers, but can't set them - at all
  • has no strftime()
  • has no valid()
  • has add, but not subtract

Date
  • can set components as integers, but can't get them as integers
  • has no time
  • has add and subtract
  • has strftime, but is requires a char array

DateTime is a beter storage format, as it can include time & timezone, but Date is easier to manipulate & display

void main() { // strftime a datetime string s = "%%a %%b %%d %%Y"; // string in storage format TimeZone tx = new TimeZone.local(); DateTime dt = new DateTime(tx, 2023, 10, 16, 12, 40, 1.0); Date dd = Date(); dd.set_year((DateYear) dt.get_year()); dd.set_month((DateMonth) dt.get_month()); dd.set_day((DateDay) dt.get_day_of_month()); char[] b = new char[64]; dd.strftime(b,s.replace("%%","%")); string ret = ""; foreach (char h in b) { ret += h.to_string(); } print("formatted datetime is %s\n",ret); print("%d\n",(1 / 5)); }
#+RESULTS:
formatted datetime is Mon Oct 16 2023 2
SUBSTRING:vala:
Note: ASCII only.

substring(x,y), where x is from and including, y is to and excluding (x+y)-1, eg:

substring(1,2) of "012345" is 1+2 is 3, -1 is 2 || "12" substring(4,2) of "012345" is 4+2 is 6, -1 is 5 || "45" substring(0,2) of "012345" is 0+2 is 2, -1 is 1 || "01"

substring(x,(y-x)), where x is from and including, y is to and excluding, eg:

substring(1,(4-1)) of "012345" is "123" substring(3,(5-3)) of "012345" is "34"
substring(x,((y-x)+1)), where x is from and including, y is to and including, eg:

substring(1,((4-1)+1)) of "012345" is "1234" substring(3,((5-3)+1)) of "012345" is "345"

To get first/last n characters:

void main (string[] args) { string s = "file.ext"; int ii = s.last_index_of("."); print("file part .....: %s\n",s.substring(0,ii)); print("extension part : %s\n",s.substring(ii)); }
#+RESULTS:
file part .....: file extension part : .ext
to get from-and-including 2 to-and-including 5 from "0123456789"
start....... 2 ( 01[23456789 ) 12^ end......... 5+1 ( 01[234567]89 ) 12 123456^ end (5+1)-start ( 01[2345]6789 ) 12 1234^ int s = 2; int e = 5; substring(s,((e+1)-s)); substring(2,6) = "234567"
substring(2,(6-2)) = "2345"


void main (string[] args) { string s = "012345"; string t = "ABCDE"; string u = "((AA + (BB / CC)) * EE)"; print("\"%s\".substring(1,2) is %s\n",s,s.substring(1,2)); print("\"%s\".substring(4,2) is %s\n",s,s.substring(4,2)); print("\"%s\".substring(1,(4-1)) is %s\n",s,s.substring(1,(4-1))); print("\"%s\".substring(3,(5-3)) is %s\n",s,s.substring(3,(5-3))); print("\"%s\".substring(1,((4-1)+1)) is %s\n",s,s.substring(1,((4-1)+1))); print("\"%s\".substring(3,((5-3)+1)) is %s\n",s,s.substring(3,((5-3)+1))); print("\"%s\".substring(2) is %s\n",s,s.substring(2)); int ii = u.last_index_of("("); string m = u.substring(ii); int oo = m.index_of(")") + 1; print("inner expression of %s is %s\n",u,u.substring(ii,oo)); }
#+RESULTS:
Compilation succeeded - 1 warning(s) "012345".substring(1,2) is 12 "012345".substring(4,2) is 45 "012345".substring(1,(4-1)) is 123 "012345".substring(3,(5-3)) is 34 "012345".substring(1,((4-1)+1)) is 1234 "012345".substring(3,((5-3)+1)) is 345 "012345".substring(2) is 2345 inner expression of ((AA + (BB / CC)) * EE) is (BB / CC)
Practical usage: extract a link from a paragraph
void main (string[] args) { string p = "some paragraph\nwith a [[val:variable]] link in it\nand [[val:another]], just to be difficult"; string c = p; string[] l = {}; int s = 0; while (c.contains("[[val:") && c.contains("]]")) { int ii = c.index_of("[[val:"); int oo = c.index_of("]]") + 2; //print("ii = %d, oo = %d\n",ii,oo); if (oo > ii) { string e = c.substring(ii,(oo - ii)); //print("\tfound link: (%s)\n",e); if (e != "") { l += e.replace("]]","").split(":")[1]; c = c.replace(e,""); //print("c = %s\n",c); } } s += 1; if (s > 10) { break; } } print("original paragraph:\n\t%s\n\n",p.replace("\n","\n\t")); print("leftover paragraph:\n\t%s\n\n",c.replace("\n","\n\t")); print("harvested linked vars:\n"); foreach (string k in l) { print("\t%s\n",k); } }
#+RESULTS:
original paragraph: some paragraph with a [[val:variable]] link in it and [[val:another]], just to be difficult leftover paragraph: some paragraph with a link in it and , just to be difficult harvested linked vars: variable another
TAKEROWS:vala:
// take rows from a 1d array, given column count // oop dresscode is for easier integration with another project // by c.p.brown 2024 public class MyDat : Object { public int cc; private string[] dat; public void clear () { dat = {}; cc = 0; } public int datlen () { return dat.length; } public void populate (string[] s) { dat = new string[s.length]; for (int i = 0; i < s.length; i++) { dat[i] = s[i]; } } public string copycell (int r, int c) { int x = (int) ((r * cc) / c); return dat[x]; } public string copyat (int x) { return dat[x]; } public void takeallbutrowsfrom (int[] p, int o) { int[] sp = p; int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; foreach (int g in gaps) { for (int i = 0; i < sp.length; i++) { int t = sp[i]; int j = i; bool cond = true; while (cond) { if (j < g) { break; } cond = (sp[j - g] < t); //if (r) { cond = !cond; } sp[j] = sp[j - g]; j -= g; } sp[j] = t; } } print("sp[0 ] = %d\n",sp[0]); clear(); cc = ops[o].cc; int a = (sp[sp.length - 1] * cc); int j = cc * p.length; int f = 0; for (int i = a; i < (ops[o].datlen() - j); i++) { int b = (i + f) / cc; while (b in sp) { print("\t skipping row %d\n",b); f += cc; b += 1; sp.resize(sp.length - 1); print("\t sp.length is %d\n",sp.length); } dat += ops[o].copyat(i+f); } } public void takerowsfrom (int[] p, int o) { int[] sp = p; int[] gaps = {693731, 270033, 133481, 7106, 3121, 1303, 701, 301, 132, 57, 23, 10, 4, 1}; foreach (int g in gaps) { for (int i = 0; i < sp.length; i++) { int t = sp[i]; int j = i; bool cond = true; while (cond) { if (j < g) { break; } cond = (sp[j - g] < t); //if (r) { cond = !cond; } sp[j] = sp[j - g]; j -= g; } sp[j] = t; } } clear(); cc = ops[o].cc; int a = (sp[sp.length - 1] * cc); int j = cc * p.length; for (int i = a; i < ops[o].datlen(); i++) { if (sp.length == 0) { break; } int b = i / cc; if (b in sp) { for (int f = 0; f < cc; f++) { dat += ops[o].copyat(i); i += 1; } i -= 1; if (sp.length > 0) { sp.resize(sp.length - 1); } } else { i += (cc - 1); } } } public string printdat (int ind, bool h) { print("cc=%d\n",cc); //int64 tts = GLib.get_monotonic_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string dlm = " | "; int[] cw = new int[cc]; int pad = "%d".printf(cc).char_count(); for (int i = 0; i < cw.length; i++) { cw[i] = pad + 3; } string o = ""; string ln = ""; // get counts 1st for (int i = 0; i < dat.length; i++) { int r = (i % cc); int cco = 0; if (dat[i] != "") { cco = dat[i].char_count() + 3; } cw[r] = int.max(cw[r],cco); } // print line for (int i = 0; i < cw.length; i++) { dlm = "-+-"; if (i == 0) { dlm = "%s+-".printf(tabs); } ln = "%s%s%*s".printf(ln,dlm,cw[i]," ").replace(" ","-"); if (i == (cw.length - 1)) { ln = "%s-+\n".printf(ln); } } o = "%s%s".printf(o,ln); if (h) { // print header for (int i = 0; i < cw.length; i++) { dlm = " | "; if (i == 0) { dlm = "%s| ".printf(tabs); } string hh = "%0*d".printf(pad,i); int cco = pad; o = "%s%s%*s%s".printf(o,dlm,(cw[i]-cco)," ",hh); if (i == (cw.length - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); } // make rows for (int i = 0; i < dat.length; i++) { int r = (i % cc); int cco = dat[i].char_count(); dlm = " | "; if (r == 0) { dlm = "%s| ".printf(tabs); } o = "%s%s%*s%s".printf(o,dlm,(cw[r]-cco)," ",dat[i]); if (r == (cc - 1)) { o = "%s |\n".printf(o); } } o = "%s%s".printf(o,ln); //int64 tte = GLib.get_monotonic_time(); //print("printdat took %.2f microseconds\n",((double) (tte - tts))); return o; } public MyDat (int l) { cc = l; } } MyDat[] ops; void main() { ops += new MyDat(3); ops[0].populate ({"0","0","0","1","1","1","2","2","2","3","3","3","4","4","4"}); print("\nops[0]:\n"); print("%s\n",ops[0].printdat(1,false)); ops += new MyDat(3); ops[1].takerowsfrom({0,3},0); print("\nops[1].takerowsfrom({0,3},0):\n"); print("%s\n",ops[1].printdat(1,false)); }
#+RESULTS:
Compilation succeeded - 1 warning(s) ops[0]: cc=3 +------+------+------+ | 0 | 0 | 0 | | 1 | 1 | 1 | | 2 | 2 | 2 | | 3 | 3 | 3 | | 4 | 4 | 4 | +------+------+------+ ops[1].takerowsfrom({0,3},0): cc=3 +------+------+------+ | 0 | 0 | 0 | | 3 | 3 | 3 | +------+------+------+
THREADS:busted::vala:

Observations on using Threadpool:
  • Both of these methods of accessing data seem fine: worker.run(ref type[] data) and new worker(type[] data) then private weak type[] data; in its properties.
  • Concurrently writing to a different items of fixed-size array seems to be fine.
  • Breaking tasks into packets may improve speed, however its inefficient if some packets finish way before the others. Assess packet numbers/sizes on a case-by-case basis.
  • Setting max_threads to -1 causes considerable delays when processing a large number of tasks, use GLib.get_num_processors() instead.
  • In various tests, Waiting for threads to complete was best handled by checking the last item in a verification checklist (int[]) set to the packet size, where each packet checks-off its entry in the array when its done.
  • The above was a dumb mistake, checking needs to be done using a packetcomplete int, incremented with the aid of mutex.
  • threadpool.unprocessed() returns the number of tasks yet to be sent to the threadpool, not the number of incomplete tasks.
  • threadpool.get_num_threads() was not a reliable measure of currently running threads.

// threadpool test // by c.p.brown 2024 // // TODO next: fix threadcheck mistake string printusecs (int64 u) { string us = "%lld h".printf((((double) u) / 3600000000)); if (u < 1000) { return "%lld μs".printf(u); } if (u < 1000000) { return "%.2f ms".printf((((double) u) / 1000.0)); } if (u < 60000000) { return "%.2f s".printf((((double) u) / 1000000.0)); } if (u < 3600000000) { return "%.2f m".printf((((double) u) / 60000000.0)); } return us; } struct packet { int[] items; } packet[] packseq (int ind, int[] x, int c, bool q, int y, int z, int u) { // x = items to package // q = use items if true, use items count if false, eg {2,12,34,43} vs {0,1,2,3} // y = max packets, usually thread-count // z = nth item to package, eg {0,1,2,3,4,5} nth 2 is {0,2,4} // u = offset items, eg {0,1,2,3,4,5} offset 2 is {2,3,4,5}, {0,1,2,3,4,5} nth 2 offset 1 is {1,3,5} packet[] o = new packet[0]; int[] srcitems = x; int maxpackets = y; int nthsrc = z; // nth int srcoffset = u; // offset int srclen = c; int[] sampleditems = {}; if (q) { srclen = srcitems.length; for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += srcitems[j]; } } else { for (int j = srcoffset; j < srclen; j += nthsrc) { sampleditems += j; } } if (sampleditems.length > y) { // y packets of n items int itemsperpacket = (sampleditems.length / maxpackets); double neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); int mxn = 0; while (neededpackets > maxpackets) { if (mxn > 1000000) { return o; } itemsperpacket += 1; neededpackets = ((double) sampleditems.length) / ((double) itemsperpacket); mxn += 1; } int np = (int) neededpackets; int dregs = sampleditems.length - (itemsperpacket * ((int) neededpackets)); int h = 0; if (dregs > 0) { h = 1; } if ((np + h) > 0) { o = new packet[(np+h)]; int f = 0; for (int i = 0; i < np; i++) { o[i] = new packet(); for (int t = 0; t < itemsperpacket; t++) { o[i].items += sampleditems[f]; f += 1; } } if (dregs > 0) { o[np] = new packet(); for (int t = 0; t < dregs; t++) { o[np].items += sampleditems[f]; f += 1; } } } } else { // one item per packet for (int k = 0; k < sampleditems.length; k++) { packet oo = new packet(); oo.items += sampleditems[k]; o += oo; } } return o; } class pnums { private int packetnum; private int[] chunks; public pnums (int n, int[] pak) { packetnum = n; chunks = pak; } public void run(ref int[] threadcheck, ref string[] seq) { foreach (int x in chunks) { seq[x] = "0"; string q = "0"; for (int i = 1; i < 100; i++) { q = "%s%d".printf(q,i); } seq[x] = q; } threadcheck[packetnum] = 1; } } class nums { private int x; public nums (int n) { x = n; } public void run(ref int[] threadcheck, ref string[] seq) { seq[x] = "0"; string q = "0"; for (int i = 1; i < 100; i++) { q = "%s%d".printf(q,i); } seq[x] = q; threadcheck[x] = 1; } } void main () { int tasks = 1000000; string[] s = new string[tasks]; int nthr = ((int) GLib.get_num_processors()); print("this computer has %d processors\n",nthr); // thread 1 task per thread, this floods the threadpool... int64 ttb = GLib.get_monotonic_time(); int scount = s.length; int[] threadcheck = new int[scount]; for (int i = 0; i < threadcheck.length; i++) { threadcheck[i] = 0; } ThreadPool<nums> tps = new ThreadPool<nums>.with_owned_data((nums) => {nums.run(ref threadcheck,ref s);},nthr,false); print("\nthreading %d tasks...\n",scount); for (int t = 0; t < scount; t++) { tps.add(new nums(t)); } print("waiting for %d tasks to complete...\n",scount); int slastindex = scount - 1; while (threadcheck[slastindex] != 1) { //GLib.Thread.usleep(1); } int64 ttf = GLib.get_monotonic_time(); print("threadpooled 1 task per thread took %s\n",printusecs(ttf-ttb)); print("\ts[%d].char_count() is %d\n",(tasks - 1),s[(tasks - 1)].char_count()); // packeted tasks per thread, nothing is queued, but there's no load-balancing string[] m = new string[tasks]; int64 tts = GLib.get_monotonic_time(); int[] mnums = new int[0]; packet[] mpackets = packseq(1,mnums,m.length,false,nthr,1,0); int mcount = mpackets.length; print("\nthreading %d packets of %d tasks...\n",mpackets.length,mpackets[0].items.length); int[] anotherthreadcheck = new int[mcount]; for (int i = 0; i < anotherthreadcheck.length; i++) { anotherthreadcheck[i] = 0; } ThreadPool<pnums> tpp = new ThreadPool<pnums>.with_owned_data((pnums) => {pnums.run(ref anotherthreadcheck, ref m);},nthr,false); for (int t = 0; t < mcount; t++) { tpp.add(new pnums(t,mpackets[t].items)); } print("waiting for %d packets to complete...\n",mcount); int lastindex = mcount - 1; while (anotherthreadcheck[lastindex] != 1) { GLib.Thread.usleep(1); } int64 tte = GLib.get_monotonic_time(); print("threadpooled %d packets of %d took %s\n",mpackets.length,mpackets[0].items.length,printusecs(tte-tts)); print("\tm[%d].char_count() is %d\n",(tasks - 1),m[(tasks - 1)].char_count()); // single threaded loop string[] k = new string[tasks]; tts = GLib.get_monotonic_time(); int klen = k.length; print("\nstarting simple for-loop of %d items...\n",klen); for (int x = 0; x < klen; x++) { k[x] = "0"; string q = "0"; for (int i = 1; i < 100; i++) { q = "%s%d".printf(q,i); } k[x] = q; } tte = GLib.get_monotonic_time(); print("forloop took %s\n",printusecs(tte-tts)); print("\t k[%d].char_count() is %d\n",(tasks - 1),k[(tasks - 1)].char_count()); //print ("k[10] is %s\n",k[10]); }
#+RESULTS:
Compilation succeeded - 7 warning(s) this computer has 16 processors threading 1000000 tasks... waiting for 1000000 tasks to complete... threadpooled 1 task per thread took 960.90 ms s[999999].char_count() is 190 threading 16 packets of 62500 tasks... waiting for 16 packets to complete... threadpooled 16 packets of 62500 took 721.33 ms m[999999].char_count() is 190 starting simple for-loop of 1000000 items... forloop took 5.55 s k[999999].char_count() is 190
TRUNCATE:vala:

Truncate a string

// truncate a string to a set length, include a tripple elipsis at the last char // by c.p.brown, 2025 string truncate(string s, int l, bool e) { StringBuilder sb = new StringBuilder(""); if ((s.char_count() - 1) > l) { int lc = l; if (e) { lc = (l - 1); } for (int i = 0; i <= lc; i++) { sb.append_unichar(s.get_char(s.index_of_nth_char(i))); } if (e) { sb.append("…"); } } else { sb.append(s); } return sb.str; } void main() { GLib.Intl.setlocale(ALL,""); int tl = 26; print("always use elipsis where text is truncated:\n"); string s = "truncate this to %d chars".printf(tl); for (int i = tl; i >= 0; i--) { s = "truncate this to %d chars".printf(i+1); print("\"%s\"\n",truncate(s,i,true)); } bool skp = false; print("use elipsis only when a word is cut:\n"); s = "truncate this to %d chars".printf(tl); for (int i = tl; i >= 0; i--) { s = "truncate this to %d chars".printf(i+1); unichar l = s.get_char(s.index_of_nth_char(i)); if (l == ' ') { skp = true; print("\"%s\"\n",truncate(s,i,false)); } else { if (skp == true) { print("\"%s\"\n",truncate(s,i,false)); skp = false; } else { print("\"%s\"\n",truncate(s,i,true)); } } } }
#+RESULTS:
always use elipsis where text is truncated: "truncate this to 27 chars" "truncate this to 26 chars" "truncate this to 25 chars" "truncate this to 24 cha…" "truncate this to 23 ch…" "truncate this to 22 c…" "truncate this to 21 …" "truncate this to 20…" "truncate this to 1…" "truncate this to …" "truncate this to…" "truncate this t…" "truncate this …" "truncate this…" "truncate thi…" "truncate th…" "truncate t…" "truncate …" "truncate…" "truncat…" "trunca…" "trunc…" "trun…" "tru…" "tr…" "t…" "…" use elipsis only when a word is cut: "truncate this to 27 chars" "truncate this to 26 chars" "truncate this to 25 chars" "truncate this to 24 cha…" "truncate this to 23 ch…" "truncate this to 22 c…" "truncate this to 21 …" "truncate this to 20 " "truncate this to 19" "truncate this to …" "truncate this to " "truncate this to" "truncate this …" "truncate this " "truncate this" "truncate th…" "truncate t…" "truncate …" "truncate " "truncate" "trunca…" "trunc…" "trun…" "tru…" "tr…" "t…" "…"
UNDO:vala:

Simple undo test using a function arg buffer.
Buffer grows as required, up to a cap, then re-uses positions.

// undo test // by c.p.brown 2024 int intmod (int l, int r) { if (l >= 0) { return (l % r); } if (l >= -r) { return (l + r); } return ((l % r) + r) % r; } int mathfunc (int a, int b, int c) { if (c == 0) { return (a + b); } if (c == 1) { return (a - b); } if (c == 2) { return (a * b); } if (c == 3) { return (a / b); } return 0; } struct mistake { int[] payload; } mistake[] mistakes; int ugh = -1; int doit (int[] a) { // grow the undo buffer as required, cap it at 5 in this case if ((ugh+1) >= mistakes.length) { if (mistakes.length < 5) { mistakes.resize(mistakes.length + 1); } } int ml = int.min(mistakes.length,5); ugh = intmod((ugh+1),ml); print("do %.2d : ",ugh); mistake ms = new mistake(); ms.payload = a; mistakes[ugh] = ms; return mathfunc(a[0],a[1],a[2]); } int undoit () { ugh = intmod((ugh-1),mistakes.length); print("undo %.2d : ",ugh); int[] a = mistakes[ugh].payload; return mathfunc(a[0],a[1],a[2]); } void main() { int m = doit({2,3,0}); string[] s = {"+","-","*","/"}; print("m = 2 + 3 = %d\n",m); m = doit({2,3,1}); print("m = 2 - 3 = %d\n",m); m = undoit(); print("m after undo is %d %s %d = %d\n",mistakes[ugh].payload[0],s[mistakes[ugh].payload[2]],mistakes[ugh].payload[1],m); m = doit({2,3,2}); print("m = 2 * 3 = %d\n",m); m = doit({2,3,3}); print("m = 2 / 3 = %d\n",m); m = doit({20,1,3}); print("m = 20 / 1 = %d\n",m); m = undoit(); print("m after undo is %d %s %d = %d\n",mistakes[ugh].payload[0],s[mistakes[ugh].payload[2]],mistakes[ugh].payload[1],m); m = undoit(); print("m after undo is %d %s %d = %d\n",mistakes[ugh].payload[0],s[mistakes[ugh].payload[2]],mistakes[ugh].payload[1],m); m = undoit(); print("m after undo is %d %s %d = %d\n",mistakes[ugh].payload[0],s[mistakes[ugh].payload[2]],mistakes[ugh].payload[1],m); m = undoit(); print("m after undo is %d %s %d = %d\n",mistakes[ugh].payload[0],s[mistakes[ugh].payload[2]],mistakes[ugh].payload[1],m); m = doit({18,2,2}); print("m = 18 * 2 = %d\n",m); m = undoit(); print("m after undo is %d %s %d = %d\n",mistakes[ugh].payload[0],s[mistakes[ugh].payload[2]],mistakes[ugh].payload[1],m); m = doit({18,4,0}); print("m = 18 + 4 = %d\n",m); m = doit({7,2,2}); print("m = 7 * 2 = %d\n",m); m = undoit(); print("m after undo is %d %s %d = %d\n",mistakes[ugh].payload[0],s[mistakes[ugh].payload[2]],mistakes[ugh].payload[1],m); }
#+RESULTS:
Compilation succeeded - 1 warning(s) do 00 : m = 2 + 3 = 5 do 01 : m = 2 - 3 = -1 undo 00 : m after undo is 2 + 3 = 5 do 01 : m = 2 * 3 = 6 do 02 : m = 2 / 3 = 0 do 03 : m = 20 / 1 = 20 undo 02 : m after undo is 2 / 3 = 0 undo 01 : m after undo is 2 * 3 = 6 undo 00 : m after undo is 2 + 3 = 5 undo 03 : m after undo is 20 / 1 = 20 do 04 : m = 18 * 2 = 36 undo 03 : m after undo is 20 / 1 = 20 do 04 : m = 18 + 4 = 22 do 00 : m = 7 * 2 = 14 undo 04 : m after undo is 18 + 4 = 22
UINT:vala:

Check behavior of uint here.

Notes:
  • uint can be used to clearly define a variable shouldn't be negative, hovever:
  • casting a negaitve int to uint will wrap it around to a huge number
  • subtracting two uints to negative will also wrap it
  • wrapped numbers can't be checked for an invalid negative number with u > -1

Conclusion: don't use uint unless you need to store a number from 2 to 4 billion without going 64-bit, or unless some other function requires/generates it, eg string.hash().

// "resize" fixed string array without overflow string[] resizestringarray (string[] o, uint l) { string[] n = new string[l]; for (uint i = 0; i < l; i++) { if (i < o.length) { n[i] = o[i]; } else { n[i] = ""; } } o = null; // this may not be necessary, still investigating... return n; } void main() { uint x = 3; uint z = 10; print("bool as uint\n"); print("((uint) (%u < 2)) * %u is %u\n",x,z,(((uint) (x < 2)) * z)); print("((uint) (%u > 2)) * %u is %u\n",x,z,(((uint) (x > 2)) * z)); print("\n"); uint y = 10; string[] s = {"zero","one","two","three","four","five","six","seven","eight","nine","ten"}; print("string[] s:\n{%s}\n",string.joinv(",",s)); print("uint s[%u] is \"%s\"\n",y,s[y]); uint n = 5; s = resizestringarray(s,n); print("s.length is now %d\n",s.length); print("uint s[(%u - 1)] is \"%s\"\n",n,s[(n - 1)]); uint r = 3; uint q = s.length / r; print("\ns.length %d (int) / %u (uint) is %u (uint)\n",s.length,r,q); q = r - n; print("%u - %u is %u\n",r,n,q); int[] ids = {0,1,2,3,4}; int sel = 4; for (int i = 0; i < ids.length; i++) { int ofs = (sel - i); print("offset is %.2f\n",(0.5 + ((20.0 * ofs) / 100.0))); } }
#+RESULTS:
bool as uint ((uint) (3 < 2)) * 10 is 0 ((uint) (3 > 2)) * 10 is 10 string[] s: {zero,one,two,three,four,five,six,seven,eight,nine,ten} uint s[10] is "ten" s.length is now 5 uint s[(5 - 1)] is "four" s.length 5 (int) / 3 (uint) is 1 (uint) 3 - 5 is 4294967294 offset is 1.30 offset is 1.10 offset is 0.90 offset is 0.70 offset is 0.50
UNICHAR2:vala:
// more unichar tests go here // by c.p.brown 2024 void main() { GLib.Intl.setlocale(ALL,""); unichar g = 'g'; unichar k = 'k'; int gn = (int) g; print("digit of g is %d\n",gn); int kn = (int) k; print("digit of k is %d\n",kn); int av = (gn + kn) / 2; print("%lc is halfway between %lc and %lc\n",((unichar) av),g,k); string s = "零1234五67«八»9零"; int lc = s.char_count() - 1; unichar l = '零'; int slc = s.char_count() - 2; unichar sl = '9'; if (s.last_index_of_char(l) == s.index_of_nth_char(lc)) { print("last char of %s is \'%s\'\n",s,s.get_char(s.index_of_nth_char(lc)).to_string()); } if (s.last_index_of_char(sl) == s.index_of_nth_char(slc)) { print("second-last char of %s is \'%s\'\n",s,s.get_char(s.index_of_nth_char(slc)).to_string()); } }
#+RESULTS:
digit of g is 103 digit of k is 107 i is halfway between g and k last char of 零1234五67«八»9零 is '零' second-last char of 零1234五67«八»9零 is '9'
UNZIP:wip::vala:

libarchive usage example is here : https:valadoc.org/libarchive/Archive.Read.html
Modify to avoid copyright issues.

UNICHAR:vala:
void main() { string nl = "\n"; unichar nlc = nl.get_char(0); bool k = true; string cr = "a row with \"a quoted \n\" in it\nand another without"; unichar n = 0; for (int i = 0; cr.get_next_char (ref i, out n);) { int s = i - 1; if (n == '\"') { k = !k; cr = cr.splice(s, i, ""); i = i - 1; } if (n == nlc && !k) { cr = cr.splice(s, i, "\\n"); } } string[] sr = cr.split("\n"); foreach (string s in sr) { print("row = %s\n",s); } }
#+RESULTS:
row = a row with a quoted \n in it row = and another without
UNIQUENAME:vala:
bool instringarray (string n, string[] h) { for (int i = 0; i < h.length; i++) { if (h[i] == n) { return true; } } return false; } bool cinstringarray (string n, string[] h) { for (int i = 0; i < h.length; i++) { if (strcmp(n,h[i]) == 0) { return true; } } return false; } void main () { string n = "name"; string[] a = { "name", "name_3", "name_03", "name_031", "name031", "name_04", "name04", "name_0301" }; int64 xtts = GLib.get_real_time(); int x = 1; string k = n; while (instringarray(k,a)) { int ld = (k.length - 1); while (ld > 0 && k[ld].isdigit()) { ld--; } string digs = k.substring((ld + 1),(k.length - (ld + 1))); k = "%s%d".printf(n.substring(0,(ld + 1)),(int.parse(digs) + 1)); x += 1; if (x > 100) { break; } } int64 xtte = GLib.get_real_time(); print("uniquename via k == a[i] loop took %f micorseconds @%d rounds and returned: %s\n",((double) (xtte - xtts)),x,k); xtts = GLib.get_real_time(); x = 1; k = n; while (cinstringarray(k,a)) { int ld = (k.length - 1); while (ld > 0 && k[ld].isdigit()) { ld--; } string digs = k.substring((ld + 1),(k.length - (ld + 1))); k = "%s%d".printf(n.substring(0,(ld + 1)),(int.parse(digs) + 1)); x += 1; if (x > 100) { break; } } xtte = GLib.get_real_time(); print("uniquename via strcmp look took %f micorseconds @%d rounds and returned: %s\n",((double) (xtte - xtts)),x,k); xtts = GLib.get_real_time(); x = 1; k = n; while ((k in a) == true) { int ld = (k.length - 1); while (ld > 0 && k[ld].isdigit()) { ld--; } string digs = k.substring((ld + 1),(k.length - (ld + 1))); k = "%s%d".printf(n.substring(0,(ld + 1)),(int.parse(digs) + 1)); x += 1; if (x > 100) { break; } } xtte = GLib.get_real_time(); print("uniquename via (k in a) took %f micorseconds @%d rounds and returned: %s\n",((double) (xtte - xtts)),x,k); }
#+RESULTS:
uniquename via k == a[i] loop took 16.000000 micorseconds @2 rounds and returned: name1 uniquename via strcmp look took 1.000000 micorseconds @2 rounds and returned: name1 uniquename via (k in a) took 1.000000 micorseconds @2 rounds and returned: name1
WRITEFILE:vala:
// write string to file, with optional backup // by c.p.brown 2023 bool spew; void backupmyfile (int ind, string f, string d) { File tq = File.new_for_path(f); string bakdirname = d; if (tq.query_exists()) { bool allgood = true; string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string basedir = Path.get_dirname(f); string basefile = Path.get_basename(f); string bakdir = basedir.concat("/",bakdirname,"/"); if (spew) { print("[I/O] %s\t backupfile is %s\n",tabs,basefile); print("[I/O] %s\t backuppath is %s\n",tabs,bakdir); } GLib.Dir bp = null; try { bp = Dir.open (bakdir, 0); } catch (Error e) { print("[I/O] %s WRITE checkdir failed: %s\n",tabs,e.message); allgood = false; } if (!allgood) { try { File tdir = File.new_for_path(bakdir); tdir.make_directory_with_parents(); } catch (Error e) { print("[I/O] %s WRITE makedirs failed: %s\n",tabs,e.message); allgood = false; } } try { bp = Dir.open (bakdir, 0); } catch (Error e) { print("[I/O] %s WRITE checkdir failed: %s\n",tabs,e.message); allgood = false; } if (allgood) { string[] ofx = basefile.split("."); string ofa = ofx[0]; string ofe = ofx[ofx.length - 1]; // find existing backups string ffn = bp.read_name(); int maxincr = 0; while (ffn != null) { if (spew) { print("[I/O] %s\t found backup file: %s\n",tabs,ffn); } string[] ffx = ffn.split("."); if (ffx.length > 1) { string ffc = ffx[ffx.length - 1]; string ffj = ffx[0]; string[] ffs = ffj.split("_"); if (ffs.length > 1) { string ffa = ffs[0]; string ffb = ffs[ffs.length - 1]; if (strcmp(ffa,ofa) == 0) { maxincr = int.max(maxincr,int.parse(ffb)); if (spew) { print("[I/O] %s\t\t backup file name part: %s\n",tabs,ffa); print("[I/O] %s\t\t backup file num part: %s\n",tabs,ffb); print("[I/O] %s\t\t backup file ext part: %s\n",tabs,ffc); } } } } ffn = bp.read_name(); } if (spew) { print("[I/O] %s\t backup max num is %d\n",tabs,maxincr); } maxincr += 1; string[]cmda = {"cp", tq.get_path(), bakdir.concat(ofa, "_", "%d".printf(maxincr), ".", ofe)}; if (spew) { print("[I/O] %s\t\t attempting to back-up %s to %s...\n",tabs,tq.get_path(),"%s%s_%d.%s".printf(bakdir,ofa,maxincr,ofe)); } try { Pid backpid; GLib.Process.spawn_async(null, cmda, null, GLib.SpawnFlags.SEARCH_PATH, null, out backpid); if (spew) { print("[I/O] %s\t\t\t file copied.\n",tabs); } } catch (Error e) { print("[I/O] %s\t\t cp spawn failed: %s\n", tabs, e.message); } } } } string writefiletosubdir (int ind, string p, string n, string e, string s, bool b) { string tabs = ("%*s").printf(ind," ").replace(" ","\t"); if (spew) { print("[I/O] %s WRITE started...\n",tabs); } if (s.strip() != "" && n.strip() != "" && e.strip() != "") { bool dobackup = b; bool allgood = true; string pth = GLib.Environment.get_current_dir(); if (p.strip() != "") { pth = pth.concat("/", p, "/"); } else { pth = pth.concat("/"); } GLib.Dir dcr = null; if (spew) { print("[I/O] %s WRITE check dir: %s\n",tabs,pth); } try { dcr = Dir.open (pth, 0); } catch (Error e) { print("[I/O] %s WRITE checkdir failed: %s\n",tabs,e.message); allgood = false; } File hfile = File.new_for_path(pth.concat(n,".",e)); File hdir = File.new_for_path(pth); if (allgood == false) { if (spew) { print("[I/O] %s WRITE make dir...\n",tabs); } try { hdir.make_directory_with_parents(); if (spew) { print("[I/O] %s WRITE made export dir: %s\n",tabs,pth); } allgood = true; } catch (Error e) { print("[I/O] %s WRITE makedirs failed: %s\n",tabs,e.message); allgood = false; } } if (allgood) { if (dobackup) { backupmyfile ((ind + 1), hfile.get_path(), "bak"); } if (spew) { print("[I/O] %s WRITE writing...\n",tabs); } try { FileOutputStream hose = hfile.replace(null,false,FileCreateFlags.PRIVATE); hose.write(s.data); if (spew) { print("[I/O] %s WRITE written to: %s\n",tabs,hfile.get_path()); } if (spew) { print("[I/O] %s WRITE ended.\n",tabs); } return hfile.get_path(); } catch (Error e) { print("[I/O] %s WRITE failed: %s\n",tabs,e.message); } } else { if (spew) { print("[I/O] %s WRITE couldn't make dir, aborting export.\n",tabs); } } } else { if (spew) { print("[I/O] %s WRITE empty input, aborting.\n",tabs); } } return ""; } void main() { spew = true; string wp = writefiletosubdir(0,"testfile", "test", "txt", "file content", true); }
#+RESULTS:
Compilation succeeded - 1 warning(s) [I/O] WRITE started... [I/O] WRITE check dir: /home/cpb/Desktop/txt/source/testfile/ [I/O] backupfile is test.txt [I/O] backuppath is /home/cpb/Desktop/txt/source/testfile/bak/ [I/O] found backup file: test_1.txt [I/O] backup file name part: test [I/O] backup file num part: 1 [I/O] backup file ext part: txt [I/O] found backup file: test_2.txt [I/O] backup file name part: test [I/O] backup file num part: 2 [I/O] backup file ext part: txt [I/O] found backup file: test_3.txt [I/O] backup file name part: test [I/O] backup file num part: 3 [I/O] backup file ext part: txt [I/O] found backup file: test_4.txt [I/O] backup file name part: test [I/O] backup file num part: 4 [I/O] backup file ext part: txt [I/O] backup max num is 4 [I/O] attempting to back-up /home/cpb/Desktop/txt/source/testfile/test.txt to /home/cpb/Desktop/txt/source/testfile/bak/test_5.txt... [I/O] file copied. [I/O] WRITE writing... [I/O] WRITE written to: /home/cpb/Desktop/txt/source/testfile/test.txt [I/O] WRITE ended.
WHIGGLE:vala:

beekaus ixlic is y laxwidq of kaeos...

void main() { string whiggle (int n) { int l = n % 10; if (l == 0) { return "th"; } int ll = n % 100; if ((ll > 10) && (ll < 20)) { return "th"; } if (l < 4) { string[] w = {"th","st","nd","rd"}; return w[l]; } return "th"; } for (int n = 1; n < 131; n++) { print("%03d%s ",n,whiggle(n)); if ((n > 0) && ((n % 10) == 0)) { print("\n"); } } }
#+RESULTS:
Compilation succeeded - 1 warning(s) 001st 002nd 003rd 004th 005th 006th 007th 008th 009th 010th 011th 012th 013th 014th 015th 016th 017th 018th 019th 020th 021st 022nd 023rd 024th 025th 026th 027th 028th 029th 030th 031st 032nd 033rd 034th 035th 036th 037th 038th 039th 040th 041st 042nd 043rd 044th 045th 046th 047th 048th 049th 050th 051st 052nd 053rd 054th 055th 056th 057th 058th 059th 060th 061st 062nd 063rd 064th 065th 066th 067th 068th 069th 070th 071st 072nd 073rd 074th 075th 076th 077th 078th 079th 080th 081st 082nd 083rd 084th 085th 086th 087th 088th 089th 090th 091st 092nd 093rd 094th 095th 096th 097th 098th 099th 100th 101st 102nd 103rd 104th 105th 106th 107th 108th 109th 110th 111th 112th 113th 114th 115th 116th 117th 118th 119th 120th 121st 122nd 123rd 124th 125th 126th 127th 128th 129th 130th
XFORMRC:vala:

Transform sequential columns into sequential rows, with gaps in the data...

// transform col sequence to rows, include placeholders for missing data // by c.p.brown 2024 void main() { string[] s = {"0A","1A","0B","1B","2B","0C","0D","1D"}; int[] rc = {2,3,1,2}; int xr = 3; for (int r = 0; r < xr; r++) { int p = r; for (int c = 0; c < rc.length; c++) { if (c > 0) { p += rc[c-1]; } if (r < rc[c]) { print("[%d] s[%d] is %s\n",r,p,s[p]); } else { print("[%d] (missing)\n",r); } } } }
#+RESULTS:
[0] s[0] is 0A [0] s[2] is 0B [0] s[5] is 0C [0] s[6] is 0D [1] s[1] is 1A [1] s[3] is 1B [1] (missing) [1] s[7] is 1D [2] (missing) [2] s[4] is 2B [2] (missing) [2] (missing)
GTKICONS:gtk4::vala:
// show gtk4 icons // by c.p.brown 2023 // icon list copied from: https://github.com/GNOME/gtk/blob/main/demos/icon-browser/icon.list // Gtk.Image.set_pixel_size() is super slow for some reason // shoudl be faster in Cairo, but heaps more work... using Gtk; int main (string[] args) { Gtk.Application wut = new Gtk.Application ("com.test.test", GLib.ApplicationFlags.DEFAULT_FLAGS); wut.activate.connect(() => { Gtk.ApplicationWindow win = new Gtk.ApplicationWindow(wut); Gtk.Box content = new Gtk.Box(VERTICAL,10); Gtk.ScrolledWindow contentscroll = new Gtk.ScrolledWindow(); Gtk.Box contentbox = new Gtk.Box(VERTICAL,10); Gtk.Entry searchentry = new Gtk.Entry(); searchentry.hexpand = true; contentscroll.vexpand = true; //contentscroll.hexpand = true; string[] icons = { "accessories-calculator", "accessories-character-map", "accessories-dictionary", "accessories-text-editor", "action-unavailable", "address-book-new", "airplane-mode", "alarm", "application-certificate", "application-exit", "application-rss+xml", "application-x-addon", "application-x-appliance", "application-x-executable", "applications-accessories", "applications-development", "applications-engineering", "applications-games", "applications-graphics", "applications-internet", "applications-multimedia", "applications-office", "applications-other", "applications-science", "applications-system", "applications-utilities", "appointment-missed", "appointment-new", "appointment-soon", "audio-card", "audio-headphones", "audio-headset", "audio-input-microphone", "audio-speakers", "audio-volume-high", "audio-volume-low", "audio-volume-medium", "audio-volume-muted", "audio-x-generic", "avatar-default", "battery", "battery-caution", "battery-caution-charging", "battery-empty", "battery-empty-charging", "battery-full", "battery-full-charged", "battery-full-charging", "battery-good", "battery-good-charging", "battery-low", "battery-low-charging", "battery-missing", "bluetooth-active", "bluetooth-disabled", "bookmark-new", "call-missed", "call-start", "call-stop", "camera-photo", "camera-video", "camera-web", "changes-allow", "changes-prevent", "channel-insecure", "channel-secure", "colorimeter-colorhug", "computer", "computer-apple-ipad", "computer-fail", "contact-new", "content-loading", "daytime-sunrise", "daytime-sunset", "dialog-error", "dialog-information", "dialog-password", "dialog-question", "dialog-warning", "display-brightness", "display-projector", "document-edit", "document-new", "document-open", "document-open-recent", "document-page-setup", "document-print", "document-print-preview", "document-properties", "document-revert", "document-save", "document-save-as", "document-send", "drive-harddisk", "drive-harddisk-ieee1394", "drive-harddisk-system", "drive-multidisk", "drive-optical", "drive-removable-media", "edit-clear", "edit-clear-all", "edit-copy", "edit-cut", "edit-delete", "edit-find", "edit-find-replace", "edit-paste", "edit-redo", "edit-select", "edit-select-all", "edit-undo", "emblem-default", "emblem-documents", "emblem-downloads", "emblem-favorite", "emblem-generic", "emblem-important", "emblem-mail", "emblem-new", "emblem-ok", "emblem-package", "emblem-photos", "emblem-readonly", "emblem-shared", "emblem-symbolic-link", "emblem-synchronized", "emblem-synchronizing", "emblem-system", "emblem-unreadable", "emblem-urgent", "emblem-videos", "emblem-web", "error-correct", "face-angel", "face-angry", "face-cool", "face-crying", "face-devilish", "face-embarrassed", "face-kiss", "face-laugh", "face-monkey", "face-plain", "face-raspberry", "face-sad", "face-shutmouth", "face-sick", "face-smile", "face-smile-big", "face-smirk", "face-surprise", "face-tired", "face-uncertain", "face-wink", "face-worried", "face-yawn", "find-location", "folder", "folder-documents", "folder-download", "folder-drag-accept", "folder-music", "folder-new", "folder-open", "folder-pictures", "folder-publicshare", "folder-remote", "folder-saved-search", "folder-templates", "folder-videos", "folder-visiting", "font-x-generic", "format-indent-less", "format-indent-more", "format-justify-center", "format-justify-fill", "format-justify-left", "format-justify-right", "format-text-bold", "format-text-direction-ltr", "format-text-direction-rtl", "format-text-italic", "format-text-strikethrough", "format-text-underline", "go-bottom", "go-down", "go-first", "go-home", "go-jump", "go-last", "go-next", "go-previous", "go-top", "go-up", "help-about", "help-browser", "help-contents", "help-faq", "image-loading", "image-missing", "image-x-generic", "input-dialpad", "input-gaming", "input-keyboard", "input-mouse", "input-tablet", "input-touchpad", "insert-image", "insert-link", "insert-object", "insert-text", "keyboard-brightness", "list-add", "list-remove", "list-remove-all", "mail-attachment", "mail-forward", "mail-mark-important", "mail-mark-junk", "mail-mark-notjunk", "mail-mark-read", "mail-mark-unread", "mail-message-new", "mail-read", "mail-replied", "mail-reply-all", "mail-reply-sender", "mail-send", "mail-send-receive", "mail-signed", "mail-signed-verified", "mail-unread", "mark-location", "media-eject", "media-flash", "media-floppy", "media-optical", "media-optical-bd", "media-optical-cd-audio", "media-optical-dvd", "media-playback-pause", "media-playback-start", "media-playback-stop", "media-playlist-consecutive", "media-playlist-repeat", "media-playlist-repeat-song", "media-playlist-shuffle", "media-record", "media-removable", "media-seek-backward", "media-seek-forward", "media-skip-backward", "media-skip-forward", "media-tape", "media-view-subtitles", "media-zip", "microphone-sensitivity-high", "microphone-sensitivity-low", "microphone-sensitivity-medium", "microphone-sensitivity-muted", "modem", "multimedia-player", "multimedia-player-apple-ipod-touch", "multimedia-volume-control", "network-cellular-3g", "network-cellular-4g", "network-cellular-acquiring", "network-cellular-connected", "network-cellular-edge", "network-cellular-gprs", "network-cellular-no-route", "network-cellular-offline", "network-cellular-signal-excellent", "network-cellular-signal-good", "network-cellular-signal-none", "network-cellular-signal-ok", "network-cellular-signal-weak", "network-cellular-umts", "network-error", "network-idle", "network-offline", "network-receive", "network-server", "network-transmit", "network-transmit-receive", "network-vpn", "network-vpn", "network-vpn-acquiring", "network-wired", "network-wired-acquiring", "network-wired-disconnected", "network-wired-no-route", "network-wired-offline", "network-wireless", "network-wireless-acquiring", "network-wireless-connected", "network-wireless-encrypted", "network-wireless-hotspot", "network-wireless-no-route", "network-wireless-offline", "network-wireless-signal-excellent", "network-wireless-signal-good", "network-wireless-signal-none", "network-wireless-signal-ok", "network-wireless-signal-weak", "network-workgroup", "night-light", "non-starred", "object-flip-horizontal", "object-flip-vertical", "object-rotate-left", "object-rotate-right", "object-select", "off", "on", "open-menu", "package-x-generic", "pan-down", "pan-end", "pan-start", "pan-up", "pda", "phone", "phone-apple-iphone", "preferences-desktop", "preferences-desktop-accessibility", "preferences-desktop-display", "preferences-desktop-font", "preferences-desktop-keyboard", "preferences-desktop-keyboard-shortcuts", "preferences-desktop-locale", "preferences-desktop-multimedia", "preferences-desktop-peripherals", "preferences-desktop-personal", "preferences-desktop-remote-desktop", "preferences-desktop-screensaver", "preferences-desktop-theme", "preferences-desktop-wallpaper", "preferences-other", "preferences-system", "preferences-system-network", "preferences-system-privacy", "preferences-system-windows", "printer", "printer-error", "printer-network", "printer-printing", "printer-warning", "process-stop", "rotation-allowed", "rotation-locked", "scanner", "security-high", "security-low", "security-medium", "semi-starred", "send-to", "software-update-available", "software-update-urgent", "star-new", "starred", "start-here", "sync-error", "sync-synchronizing", "system-file-manager", "system-help", "system-lock-screen", "system-log-out", "system-reboot", "system-run", "system-search", "system-shutdown", "system-software-install", "system-software-update", "system-users", "tab-new", "task-due", "task-past-due", "text-html", "text-x-generic", "text-x-generic-template", "text-x-preview", "text-x-script", "tools-check-spelling", "touchpad-disabled", "trophy-bronze", "trophy-gold", "trophy-silver", "uninterruptible-power-supply", "user-available", "user-away", "user-bookmarks", "user-busy", "user-desktop", "user-home", "user-idle", "user-info", "user-invisible", "user-offline", "user-status-pending", "user-trash", "user-trash-full", "utilities-system-monitor", "utilities-terminal", "video-display", "video-x-generic", "view-continuous", "view-dual", "view-fullscreen", "view-grid", "view-list", "view-more", "view-paged", "view-pin", "view-refresh", "view-restore", "view-sort-ascending", "view-sort-descending", "view-wrapped", "weather-clear", "weather-clear-night", "weather-few-clouds", "weather-few-clouds-night", "weather-fog", "weather-overcast", "weather-severe-alert", "weather-showers", "weather-showers-scattered", "weather-snow", "weather-storm", "weather-windy", "window-close", "window-maximize", "window-minimize", "window-new", "window-restore", "x-office-address-book", "x-office-calendar", "x-office-document", "x-office-document-template", "x-office-presentation", "x-office-presentation-template", "x-office-spreadsheet", "x-office-spreadsheet-template", "x-package-repository", "zoom-fit-best", "zoom-in", "zoom-original", "zoom-out" }; int washeight = 500; for (int i = 0; i < icons.length; i++) { Gtk.Image ib = new Gtk.Image.from_icon_name(icons[i]); ib.set_pixel_size(win.default_height / 5); Gtk.Label ilbl = new Gtk.Label(icons[i]); ilbl.selectable = true; Gtk.Box ibx = new Gtk.Box(HORIZONTAL,10); ibx.append(ib); ibx.append(ilbl); contentbox.append(ibx); } searchentry.changed.connect(() => { if (searchentry.text.strip() != "") { while (contentbox.get_first_child() != null) { contentbox.remove(contentbox.get_first_child()); } for (int i = 0; i < icons.length; i++) { if (icons[i].contains(searchentry.text)) { Gtk.Image ib = new Gtk.Image.from_icon_name(icons[i]); ib.set_pixel_size(win.default_height / 5); Gtk.Label ilbl = new Gtk.Label(icons[i]); ilbl.selectable = true; Gtk.Box ibx = new Gtk.Box(HORIZONTAL,10); ibx.append(ib); ibx.append(ilbl); contentbox.append(ibx); } } } else { for (int i = 0; i < icons.length; i++) { Gtk.Image ib = new Gtk.Image.from_icon_name(icons[i]); ib.set_pixel_size(win.default_height / 5); Gtk.Label ilbl = new Gtk.Label(icons[i]); ilbl.selectable = true; Gtk.Box ibx = new Gtk.Box(HORIZONTAL,10); ibx.append(ib); ibx.append(ilbl); contentbox.append(ibx); } } }); win.notify.connect(() => { if (((int) win.default_height) != washeight) { Gtk.Box bb = (Gtk.Box) contentbox.get_first_child(); while (bb != null) { Gtk.Image ii = (Gtk.Image) bb.get_first_child(); ii.set_pixel_size(win.default_height / 5); bb = (Gtk.Box) bb.get_next_sibling(); } washeight =((int) win.default_height); } }); contentscroll.set_child(contentbox); content.append(searchentry); content.append(contentscroll); win.set_child(content); win.default_width = 500; win.default_height = 500; win.present(); }); return wut.run(args); }
TBLFM PARSER:vala:
// parse and run org tblfm expressions // by c.p.brown, 2023 // // todo next: finish elisp functions bool spew; int imod (int a, int b) { if (a >= 0) { return (a % b); } if (a >= -b) { return (a + b); } return ((a % b) + b) % b; } string[,] orgtodat (int ind, string org) { int64 odtts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string[,] dat = {{""}}; string[] rr = org.split("\n"); if (rr[0].has_prefix("|")) { int ii = rr[0].index_of("|"); int oo = rr[0].last_index_of("|"); string headrow = rr[0]; if(oo > (ii + 1)) { headrow = rr[0].substring((ii+1),(oo - (ii + 1))); } string[] hh = headrow.split("|"); string[] headers = {}; for (int h = 0; h < hh.length; h++) { if(hh[h].strip() != "") { headers += hh[h].strip(); } } int num_rows = 0; int num_columns = headers.length; for (int r = 0; r < rr.length; r++) { if (rr[r] != null && rr[r].strip() != "" && rr[r].has_prefix("|") == true) { num_rows += 1; } } dat = new string[num_rows,num_columns]; int tr = 0; for (int r = 0; r < rr.length; r++) { if (rr[r] != null && rr[r].strip() != "" && rr[r].has_prefix("|") == true) { ii = rr[r].index_of("|"); oo = rr[r].last_index_of("|"); if (oo > (ii + 1)) { rr[r] = rr[r].substring((ii+1),(oo - (ii + 1))); } string[] cc = rr[r].split("|"); if (cc.length == 1 && cc[0].contains("-+-")) { cc = rr[r].split("+"); } for (int c = 0; c < num_columns; c++) { dat[tr,c] = cc[c].strip(); } tr += 1; } } } int64 odtte = GLib.get_real_time(); if (spew) { print("%sorgtodat took %f microseconds\n",tabs,((double) (odtte - odtts)));} return dat; } string reorgtable (int ind, string[,] dat) { int64 reots = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); int[] maxlen = new int[dat.length[1]]; string o = ""; string hln = "------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------"; for (int m = 0; m < maxlen.length; m++) { maxlen[m] = 0; } for (int r = 0; r < dat.length[0]; r++) { for (int c = 0; c < dat.length[1]; c++) { string lc = dat[r,c].replace("-",""); if (lc.strip().length == 0) { continue; } maxlen[c] = int.max(maxlen[c],dat[r,c].length); } } for (int r = 0; r < dat.length[0]; r++) { bool ishline = false; string hc = dat[r,0].replace("-","").strip(); if (hc.length == 0) { for (int c = 1; c < dat.length[1]; c++) { hc = hc.concat(dat[r,c]); } hc = hc.replace("-","").strip(); if (hc.length == 0) { ishline = true; } } if (ishline) { o = o.concat("|"); for (int c = 0; c < (dat.length[1] - 1); c++) { //print("%.*s%s\n",5,s,"heading"); o = "%s-%.*s%s+".printf(o,maxlen[c],hln,"-"); } o = "%s-%.*s%s|\n".printf(o,maxlen[(dat.length[1] - 1)],hln,"-"); } else { o = o.concat("| "); for (int c = 0; c < dat.length[1]; c++) { o = "%s%-*s | ".printf(o,maxlen[c],dat[r,c]); } o._chomp(); o = o.concat("\n"); } } int64 reote = GLib.get_real_time(); if (spew) { print("%sreorgtable took %f microseconds\n",tabs,((double) (reote - reots)));} return o; } // TODO: handle pre-trimmed input... int getrefindex (int ind, int myi, string r, string[,] dat) { int64 idxts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); int o = 0; if (r != null && r.strip() != "") { if (spew) { print("%sgetrefindex: input is %s\n",tabs,r); } string s = r; s.canon("1234567890<>I",'.'); int oo = s.index_of("."); if (oo > 0) { s = s.substring(0,oo); switch (s.get_char(0)) { case '>': if (spew) { print("%s\tget prev ref (>)...\n",tabs); } o = (myi - (s.split(">").length - 1)); break; case '<': if (spew) { print("%s\tget next ref (<)...\n",tabs); } o = (myi + (s.split("<").length - 1)); break; case 'I': if (spew) { print("%s\tget hline ref (I)...\n",tabs); } int qq = 0; int x = s.split("I").length - 1; for (int i = 0; i < dat.length[0]; i++) { if (dat[i,0].has_prefix("--")) { qq += 1; if (qq == x) { o = i + 1; break; } } } break; default: o = int.parse(s) - 1; break; } } else { int t = 0; if (int.try_parse(s,out t)) { o = t - 1; } } } if (spew) { print("%sgetrefindex: zero-based cell ref is %d\n",tabs,o); } int64 idxte = GLib.get_real_time(); if (spew) { print("%sgetrefindex: took %f microseconds\n",tabs,((double) (idxte - idxts)));} return o; } // separate subtraction (a - b) from negative (-a) string subminus (int ind, string s) { int64 pusts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string o = s; if (s.contains("-")) { char[] nums = {'0','1','2','3','4','5','6','7','8','9'}; for (int h = 0; h < s.length; h++) { if (s[h] == '-') { if (h > 0 && h < (s.length - 1)) { if (s[(h-1)] in nums && s[(h+1)] in nums) { o = o.splice(h,(h+1),"!"); } else { // 3-1 if (s[(h-1)] == ')' && s[(h+1)] in nums) { o = o.splice(h,(h+1),"!"); } else { // )-1 if (s[(h-1)] == ')' && s[(h+1)] == '(') { o = o.splice(h,(h+1),"!"); } else { // )-( if (s[(h-1)] in nums && s[(h+1)] == ' ') { o = o.splice(h,(h+1),"!"); } else { // 3- if (s[(h-1)] in nums && s[(h+1)] == '(') { o = o.splice(h,(h+1),"!"); } else { // 3-( if (s[(h-1)] == ')' && s[(h+1)] == ' ') { o = o.splice(h,(h+1),"!"); } else { // )- if (s[(h-1)] == ' ' && s[(h+1)] == ' ') { o = o.splice(h,(h+1),"!"); } // - } } } } } } } } } } int64 puste = GLib.get_real_time(); if (spew) { print("%ssubminus took %f microseconds\n",tabs,((double) (puste - pusts)));} return o; } string replacerefs (int ind, int myr, int myc, string inner, string[,] tbldat) { int64 refts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); if (inner != null && inner.strip() != "") { if (inner.contains("@") || inner.contains("$")) { if (spew) { print("%sreplacerefs: input is %s\n",tabs,inner); } string s = inner; int[] rc = {-1,-1}; int y = 0; int b = 0; int r = myr; int c = myc; while (s.contains("@") || s.contains("$")) { if (y > 100) {break;} string t = s; t.canon("$@1234567890<>I",'.'); //print("\tevallisp: s.canon = %s\n",t); for (int h = b; h < t.length; h ++) { //print("evallisp: \tchecking char %d %c in block starting at %d\n",h,t[h],b); if (t[h] == '.' || (h == t.length - 1)) { if ((rc[0] + rc[1]) != -2) { //print("evallisp: \t\trc[0] = %d, rc[1] = %d\n",rc[0],rc[1]); if (rc[0] == -1) { r = myr; rc[0] = 99999; } if (rc[1] == -1) { c = myc; rc[1] = 99999;} if(spew) { print("%s\tr = %d, c = %d\n",tabs,r,c); } s = s.splice(int.min(rc[0],rc[1]),h,tbldat[r,c]); if (spew) { print("%sreplacerefs: spliced expression: %s\n",tabs,s); } } rc = {-1,-1}; b = ((h + 1) - (t.length - s.length)); r = myr; c = myc; break; } if (t[h] == '$') { string cs = t.substring((h+1)); //print("\tevallisp: \t\tcs = %s\n",cs); rc[1] = h; c = getrefindex((ind + 1),myc,cs, tbldat); //print("\tevallisp: \t\tc = %d\n",c); } if (t[h] == '@') { string rs = t.substring((h + 1)); //print("\tevallisp: \t\trs = %s\n",rs); rc[0] = h; r = getrefindex((ind + 1),myr,rs, tbldat); //print("\tevallisp: \t\tr = %d\n",r); } } y += 1; } int64 refte = GLib.get_real_time(); if (spew) { print("%sreplacerefs took %f microseconds\n",tabs,((double) (refte - refts)));} return s; } } int64 refte = GLib.get_real_time(); if (spew) { print("%sreplacerefs took %f microseconds\n",tabs,((double) (refte - refts)));} return inner; } string evalmaths (int ind, int myr, int myc, string inner, string[,] tbldat) { int64 mthts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string o = inner; string[] ops = {"*", "/", "+", "!"}; if (inner != null && inner.strip() != "") { if (spew) { print("%sdomaths input .....: %s\n",tabs,inner); } string s = inner; if (inner.contains("@") || inner.contains("$")) { s = replacerefs((ind + 1),myr, myc, inner, tbldat); } if (s.contains("-")) { s = subminus((ind + 1),s); s = s.replace("-"," -"); } if (spew) { print("%sdomaths expression : %s\n",tabs,s); } int y = 0; foreach (string x in ops) { //if (s.contains(x)) { while (s.contains(x)) { //print("domaths: \texpression contains %s\n",x); if (y > 10) { break; } double sm = 0.0; string t = s; switch (x) { case "*": t.canon("1234567890.*-",'_'); break; case "/": t.canon("1234567890./-",'_'); break; case "+": t.canon("1234567890.+-",'_'); break; case "!": t.canon("1234567890.!-",'_'); break; default: t.canon("1234567890.",'_'); break; } //print("domaths: \ts.canon: %s\n",t); string[] sp = t.split(x); if (sp.length > 1) { //print("domaths: \tleft = %s, right = %s\n",sp[0],sp[1]); int aii = 0; int oo = sp[1].length - 1; int splen = sp[0].length; if (sp[0].length > 0 && sp[0].contains("_")) { for ( int h = (sp[0].length - 1); h >= 0; h--) { //print("domaths: \tleft tail search at %d: %c == %c ?\n",h,sp[0][h],'_'); if (sp[0][h] != '_') { oo = h; break; } } for ( int h = oo; h >= 0; h--) { //print("domaths: \tleft head search at %d: %c == %c ?\n",h,sp[0][h],'_'); if (sp[0][h] == '_') { aii = h + 1; break; } } //if (oo > 0 && oo < (sp[0].length - 2)) { oo += 1; } //print("domaths: left starts at %d, ends at %d\n",aii,oo); if (aii < sp[0].length && aii < oo && oo < sp[0].length) { sp[0] = sp[0].substring(aii,(oo - aii + 1)); //print("domaths: left substring(%d,%d): %s\n",aii,(oo - aii),sp[0]); } } int ii = 0; if (sp[1].length > 0 && sp[1].contains("_")) { for ( int h = (sp[1].length - 1); h >= 0; h--) { if (sp[1][h] != '_') { oo = h; break; } } for ( int h = oo; h >= 0; h--) { if (sp[1][h] == '_') { ii = h + 1; break; } } if (ii < sp[1].length && ii < oo && oo < sp[1].length) { sp[1] = sp[1].substring(ii,(oo - ii + 1)); //print("domaths: right substring(%d,%d): %s\n",ii,(oo - ii + 1),sp[1]); } } //print("domaths: \tleft = %s, right = %s\n",sp[0],sp[1]); oo = oo + splen + 2; double aa = 0.0; double bb = 0.0; if (double.try_parse(sp[0].strip(),out aa)) { if (double.try_parse(sp[1].strip(),out bb)) { switch (x) { case "*": sm = aa * bb; break; case "/": sm = aa / bb; break; case "+": sm = aa + bb; break; case "!": sm = aa - bb; break; default: sm = 0.0; break; } } else { print("%sERROR: %s or %s are not float",tabs,sp[0],sp[1]); break; } } else { print("%sERROR: %s or %s are not float",tabs,sp[0],sp[1]); break; } //print("domaths: s.length: %d\n",s.length); //print("domaths: \tsplicing from %d to %d\n",aii,oo); if (aii >= 0 && aii < s.length && aii < oo) { if (oo > aii && oo <= s.length) { s = s.splice(aii,oo,"%f".printf(sm)); print("%sdomaths splice ....: %s\n",tabs,s); } } y += 1; } else { break; } } } //} o = s; } int64 mthte = GLib.get_real_time(); if (spew) { print("%sdomaths took %f microseconds\n",tabs,((double) (mthte - mthts)));} return o; } string doformat (int ind, string n) { int64 frmts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); if (n != null && n != "") { if (spew) { print("%sdoformat: input is %s\n",tabs,n); } string[] np = n.split(";"); if (np.length == 2) { if (np[0] != "" && np[1] != "") { string h = np[1].printf(double.parse(np[0])); int64 frmte = GLib.get_real_time(); if (spew) { print("%sdoformat took %f microseconds\n",tabs,((double) (frmte - frmts)));} return h; } } } int64 frmte = GLib.get_real_time(); if (spew) { print("%sdoformat took %f microseconds\n",tabs,((double) (frmte - frmts)));} return n; } string evallisp (int ind, int myr, int myc, string instr, string[,] tbldat) { int64 lspts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); string inner = instr; if (inner != null && inner.strip() != "") { if (spew) { print("%sevallisp: input is %s\n",tabs,inner); } int ic = 1; int ii = -1; if (inner.contains("format")) { if (spew) { print("%s\tformat...\n",tabs); } inner = inner.replace("format",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.split(" "); int ptl = 0; string[] k = {}; foreach (string g in pts) { if (g.strip() != "") { ptl += 1; k += g.strip(); } } if (k.length > 1 && k[0].contains("%")) { if (spew) { print("%s\t\tgetting tokens in %s\n",tabs,k[0]); } int n = 1; int ival = 0; double dval = 0.0; k[0] = k[0].replace("%","%%"); int y = 0; while (k[0].contains("%")) { if (y > 10) { break; } ii = k[0].index_of("%"); string tk = k[0].substring(ii,3); if (strcmp(tk,"%%d") == 0) { if (int.try_parse(k[n],out ival)) { k[0] = k[0].splice(ii,(ii+3),k[n]); if (spew) { print("%s\t\tspliced format: %s\n",tabs,k[0]); } n += 1; } else { int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp format took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "ERROR: format arg %d not an int".printf(n); } } if (strcmp(tk,"%%f") == 0) { if (double.try_parse(k[n],out dval)) { k[0] = k[0].splice(ii,(ii+3),k[n]); if (spew) { print("%s\t\tspliced format: %s\n",tabs,k[0]); } n += 1; } else { int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp format took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "ERROR: format arg %d not an int".printf(n); } } if (strcmp(tk,"%%s") == 0) { k[0] = k[0].splice(ii,(ii+3),k[n]); if (spew) { print("%s\t\tspliced format: %s\n",tabs,k[0]); } n += 1; } y += 1; } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp format took %f microseconds\n",tabs,((double) (lspte - lspts)));} return k[0]; } } if (inner.contains("make-string")) { // (make-string 5 ?x) if (spew) { print("%s\tmake-string...\n",tabs); } inner = inner.replace("make-string",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.split(" "); if (pts.length > 1 && pts[0] != "") { bool docount = false; for (int h = 0; h < pts.length; h++) { string hh = pts[h].replace("\"","").strip(); if (pts[h].has_prefix("?") == false) { pts[0] = "%.*s".printf(ic,hh); } ic = 1; docount = int.try_parse(hh,out ic); } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp make-string took %f microseconds\n",tabs,((double) (lspte - lspts)));} return string.joinv(" ",pts); } return "ERROR: malformed make-string expression"; } if (inner.contains("substring")) { // (substring ?a ?b ?c...) if (spew) { print("%s\tsubstring...\n",tabs); } inner = inner.replace("substring",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.split(" "); if (pts.length > 1 && pts[0] != "") { string sp = pts[0].replace("\"","").strip(); int si = 0; if (int.try_parse(pts[1],out si)) { if (pts.length > 2) { int so = 0; if (int.try_parse(pts[2],out so)) { if (si >= 0 && so > si) { sp = sp.substring(si,so); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp substring took %f microseconds\n",tabs,((double) (lspte - lspts)));} return sp; } } } else { if (si >= 0) { sp = sp.substring(si); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp substring took %f microseconds\n",tabs,((double) (lspte - lspts)));} return sp; } } } } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp substring took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "ERROR: malformed substring expression"; } if (inner.contains("string")) { // (string ?a ?b ?c...) if (spew) { print("%s\tstring...\n",tabs); } inner = inner.replace("string",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.split(" "); if (pts.length > 0 && pts[0] != "") { for (int h = 0; h < pts.length; h++) { if (pts[h].has_prefix("?")) { string hh = pts[h].replace("\"","").strip(); hh = hh.replace("?",""); pts[h] = "%s".printf(hh); } } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp string took %f microseconds\n",tabs,((double) (lspte - lspts)));} return string.joinv(" ",pts); } return "ERROR: malformed string expression"; } if (inner.contains("concat")) { // (concat "s" "s") if (spew) { print("%s\tconcat...\n",tabs); } inner = inner.replace("concat",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.split(" "); if (pts.length > 0 && pts[0] != "") { for (int h = 0; h < pts.length; h++) { string hh = pts[h].replace("\"","").strip(); pts[h] = hh; } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp concat took %f microseconds\n",tabs,((double) (lspte - lspts)));} return string.joinv("",pts); } return "ERROR: malformed concat expression"; } if (inner.contains("downcase")) { // (downcase "s") if (spew) { print("%s\tdowncase...\n",tabs); } inner = inner.replace("downcase",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp downcase took %f microseconds\n",tabs,((double) (lspte - lspts)));} return inner.down(); } if (inner.contains("upcase")) { // (upcase "s") if (spew) { print("%s\tupcase...\n",tabs); } inner = inner.replace("upcase",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp upcase took %f microseconds\n",tabs,((double) (lspte - lspts)));} return inner.up(); } // number if (inner.contains("abs")) { // (abs -1) if (spew) { print("%s\tabs...\n",tabs); } inner = inner.replace("abs",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = v.abs(); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp abs took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed abs expression"; } if (inner.contains("mod")) { // (mod 9 4) if (spew) { print("%s\tmod...\n",tabs); } inner = inner.replace("mod",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 2 && pts[0] != "") { double uu = 0.0; if (double.try_parse(pts[0],out uu)) { double vv = 0.0; if (double.try_parse(pts[1],out(vv))) { if (spew) { print("%s\tfmod(%f,%f)\n",tabs,uu,vv); } uu = Math.fmod(uu,vv); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp fmod took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(uu); } } int u = 0; if (int.try_parse(pts[0], out u)) { int v = 0; if (int.try_parse(pts[1], out v)) { u = imod(u,v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp mod took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%d".printf(u); } } } return "ERROR: malformed mod expression"; } if (inner.contains("random")) { if (spew) { print("%s\trand...\n",tabs); } inner = inner.replace("random",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1) { int u = 0; if (int.try_parse(pts[0],out u)) { GLib.Rand rnd = new GLib.Rand(); int64 lspte = GLib.get_real_time(); rnd.set_seed(((int32) lspte)); if (spew) { print("%sevallisp random took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%d".printf(rnd.int_range(0,((int32) u))); } } return "ERROR: malformed random expression"; } if (inner.contains("ceiling")) { // (ceiling 1.5) if (spew) { print("%s\tceiling...\n",tabs); } inner = inner.replace("ceiling",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.ceil(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp ceiling took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed ceiling expression"; } if (inner.contains("floor")) { // (floor 1.5) if (spew) { print("%s\tfloor...\n",tabs); } inner = inner.replace("floor",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.floor(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp floor took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed floor expression"; } if (inner.contains("round")) { // (round 1.5) if (spew) { print("%s\tround...\n",tabs); } inner = inner.replace("round",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.round(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp round took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed round expression"; } if (inner.contains("truncate")) { // (truncate 1.5) if (spew) { print("%s\ttruncate...\n",tabs); } inner = inner.replace("truncate",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.trunc(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp truncate took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed truncate expression"; } if (inner.contains("min")) { // (min 1 2 3...) if (spew) { print("%s\tmin...\n",tabs); } inner = inner.replace("min",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length > 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { for (int h = 1; h < pts.length; h++) { double j = 0.0; if (double.try_parse(pts[h],out j)) { v = double.min(v,j); } } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed min expression"; } if (inner.contains("max")) { // (max 1 2 3...) if (spew) { print("%s\tmax...\n",tabs); } inner = inner.replace("max",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length > 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { for (int h = 1; h < pts.length; h++) { double j = 0.0; if (double.try_parse(pts[h],out j)) { v = double.max(v,j); } } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed max expression"; } if (inner.contains("expt")) { // (expt 2.0 1.2) if (spew) { print("%s\texpt...\n",tabs); } inner = inner.replace("expt",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length > 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { for (int h = 1; h < pts.length; h++) { double j = 0.0; if (double.try_parse(pts[h],out j)) { v = Math.pow(v,j); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp expt took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } } } return "ERROR: malformed expt (pow) expression"; } if (inner.contains("exp")) { // (exp 1.5) if (spew) { print("%s\texp...\n",tabs); } inner = inner.replace("exp",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.exp(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp exp took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed exp expression"; } if (inner.contains("log")) { // (log 1.5) if (spew) { print("%s\tlog...\n",tabs); } inner = inner.replace("log",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.log(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp log took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed log expression"; } if (inner.contains("asin")) { // (asin 1.5) if (spew) { print("%s\tasin...\n",tabs); } inner = inner.replace("asin",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.asin(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp asin took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed asin expression"; } if (inner.contains("acos")) { // (acos 1.5) if (spew) { print("%s\tacos...\n",tabs); } inner = inner.replace("acos",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.acos(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp acos took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed acos expression"; } if (inner.contains("atan")) { // (atan 1.5) if (spew) { print("%s\tatan...\n",tabs); } inner = inner.replace("atan",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.atan(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp atan took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed atan expression"; } if (inner.contains("sin")) { // (sin 1.5) if (spew) { print("%s\tsin...\n",tabs); } inner = inner.replace("sin",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.sin(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp sin took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed sin expression"; } if (inner.contains("cos")) { // (cos 1.5) if (spew) { print("%s\tcos...\n",tabs); } inner = inner.replace("cos",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.cos(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp cos took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed cos expression"; } if (inner.contains("tan")) { // (tan 1.5) if (spew) { print("%s\ttan...\n",tabs); } inner = inner.replace("tan",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.tan(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp tan took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed tan expression"; } if (inner.contains("sqrt")) { // (sqrt 1.5) if (spew) { print("%s\tsqrt...\n",tabs); } inner = inner.replace("sqrt",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); string[] pts = inner.strip().split(" "); if (pts.length == 1 && pts[0] != "") { double v = 0.0; if (double.try_parse(pts[0], out v)) { v = Math.sqrt(v); int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp sqrt took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } return "ERROR: malformed sqrt expression"; } if (inner.contains("float-pi")) { // (float-pi) if (spew) { print("%s\tfloat-pi...\n",tabs); } inner = inner.replace("float-pi",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); double v = Math.PI; int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp PI took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } if (inner.contains("float-e")) { // (float-e) if (spew) { print("%s\tfloat-e...\n",tabs); } inner = inner.replace("float-e",""); inner = inner.replace("(",""); inner = inner.replace(")","").strip(); double v = Math.E; int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp E took %f microseconds\n",tabs,((double) (lspte - lspts)));} return "%f".printf(v); } } int64 lspte = GLib.get_real_time(); if (spew) { print("%sevallisp took %f microseconds\n",tabs,((double) (lspte - lspts)));} return instr; } double dosum (int ind, int myr, int myc, string inner, string[,] tbldat) { int64 sumts = GLib.get_real_time(); string tabs = ("%*s").printf(ind," ").replace(" ","\t"); double sm = 0.0; if (inner != null && inner.strip() != "") { if (spew) { print("%sdosum: %s\n",tabs,inner); } string[] sp = inner.split(".."); if (sp.length == 2) { if (sp[0].strip() != "" && sp[1].strip() != "") { int[] coords = {-1,-1,-1,-1}; for (int x = 0; x < 2; x++) { int b = 0; int[] rc = {-1,-1}; int y = 0; int r = myr; int c = myc; while (sp[x].contains("@") || sp[x].contains("$")) { if (y > 200) { print("ERROR: reference string stuck in the loop: %s\n",sp[x]); break; } string t = sp[x]; t.canon("$@1234567890<>I",'.'); for (int h = b; h < t.length; h ++) { if (t[h] == '.' || (h == t.length - 1)) { if ((rc[0] + rc[1]) != -2) { if (rc[0] == -1) { r = myr; rc[0] = 99999; } if (rc[1] == -1) { c = myc; rc[1] = 99999; } coords[(x+x)] = r; coords[(1+(x+x))] = c; sp[x] = sp[x].splice(int.min(rc[0],rc[1]),h,""); if (spew) { print("%sdosum: part%d row = %d, col = %d\n",tabs,x,r,c); } } rc = {-1,-1}; b = ((h + 1) - (t.length - sp[x].length)); r = myr; c = myc; break; } if (t[h] == '$') { string cs = t.substring((h+1)); rc[1] = h; c = getrefindex((ind + 1),myc,cs, tbldat); } if (t[h] == '@') { string rs = t.substring((h + 1)); rc[0] = h; r = getrefindex((ind + 1),myr,rs, tbldat); } } y += 1; } } if (spew) { print("%sdosum: zero-based cell refs...\n",tabs); } if (spew) { print("%s\trow1=%d, col1=%d\n%s\trow2=%d, col2=%d\n",tabs,coords[0],coords[1],tabs,coords[2],coords[3]); } if (coords[0] == coords[2]) { for (int i = coords[1]; i <= coords[3]; i++) { double dd = 0.0; if (double.try_parse(tbldat[coords[0],i])) { if ( dd != 0.0) { sm += dd; } } } if (spew) { print("%s\thsum = %f\n",tabs,sm); } } if (coords[1] == coords[3]) { for (int i = coords[0]; i <= coords[2]; i++) { double dd = 0.0; if (double.try_parse(tbldat[i,coords[1]],out dd)) { if ( dd != 0.0) { sm += dd; } } } if (spew) { print("%s\tvsum = %f\n",tabs,sm); } } } } } int64 sumte = GLib.get_real_time(); if (spew) { print("%sdosum took %f microseconds\n",tabs,((double) (sumte - sumts)));} return sm; } string doelisp (int ind, int r, int c, string e, string[,] tbldat) { string ret = e; int y = 0; while (ret.contains("'(")) { if (spew) { print("\t\telisp: lisp expression is %s\n",e); } if (y > 200) { print("ERROR: expression stuck in a loop: %s\n",e); break; } string o = e; int qii = e.index_of("'(") + 1; int qoo = -1; int oc = 0; // match brace of elisp ret = ret.splice((qii - 1),qii," "); if (spew) { print("\t\telisp: spliced comma: %s\n",e); } for (int h = qii; h < o.length; h++) { if (o[h] == '(') { if (h == qii) { oc = 1; } else { oc += 1; } } if (o[h] == ')') { oc -= 1; qoo = h; if ( oc == 0 ) { break; } } } // isolate elisp o = e.substring(qii,(qoo - (qii - 1))); if (spew) { print("\t\telisp: outer lisp expression is %s\n",o); } int z = 0; // sub-expressions while (o.contains("(")) { if (z > 200) { print("\nERROR: expression stuck in the elisp inner loop: %s\n\n",o); break; } // incasement if (spew) { print("\t\t\telisp inner: iteration %d\n",z); } int eii = 0; int eoo = -1; eii = o.last_index_of("("); string m = o.substring(eii); eoo = m.index_of(")") + 1; if (eoo != -1) { string inner = o.substring(eii,eoo); if (inner.contains("@") || inner.contains("$")) { inner = replacerefs(4,r, c, inner, tbldat); } if (spew) { print("\t\t\telisp inner: lisp expression: %s\n",inner); } string em = evallisp(4,r,c,inner,tbldat); if ( em == inner ) { em = em.replace("(",""); em = em.replace(")",""); em = "ERROR: unknown function %s".printf(em); } o = o.splice(eii,(eoo + eii),em); o = o.replace("\"",""); if (spew) { print("\t\t\telisp inner: spliced expression = %s\n",o); } } else { break; } z += 1; } ret = ret.splice(qii,qoo+1,o); } return ret; } string domaths (int ind, int r, int c, string e, string[,] tbldat) { string ret = e; if ( e.strip() != "") { string o = e; int z = 0; int tii = -1; int too = -1; string inner = e; if (spew) { print("\ttblfm checking expression: %s\n",e); } while (o.contains("(")) { if (z > 200) { print("\nERROR: expression stuck in the tblfm inner loop: %s\n\n",o); break; } tii = o.last_index_of("("); string m = o.substring(tii); too = m.index_of(")") + 1; inner = o.substring(tii,too); if (spew) { print("\t\ttblfm: inner expression: %s\n",inner); } if (inner.contains("..")) { m = o.substring(0,tii); double sm = dosum(3,r,c,inner,tbldat); tii = m.last_index_of("vsum"); if (spew) { print("\t\ttblfm: sum = %f\n",sm); } o = o.splice(tii,(too + tii + 4),"%f".printf(sm)); if (spew) { print("\t\ttblfm: spliced expression = %s\n",o); } } if (inner.contains("/") || inner.contains("*") || inner.contains("+") || inner.contains("-")){ inner = inner.replace("(",""); inner = inner.replace(")",""); string sm = evalmaths(3,r, c, inner, tbldat); if (spew) { print("\t\ttblfm: result = %s\n",sm); } o = o.splice(tii,(too + tii),sm); if (spew) { print("\t\ttblfm: spliced expression = %s\n",o); } } if (o == e) { o = "ERROR: unknown function %s".printf(o); break; } z += 1; } if(o != null && o.strip() != "") { ret = o; } } return ret; } void main() { int64 ofmts = GLib.get_real_time(); spew = true; string[,] dat; string orgtbl = """| AA | BB | CC | DD |\n |--------+-------+--------+----------|\n | 68.0 | 39.47 | 128.15 | 337.59 |\n | 403.88 | 16.21 | 117.03 | -9.0 |\n | 5.8 | 14.73 | 58.1 | 107.64 |\n |--------+-------+--------+----------|\n | | | | |"""; dat = orgtodat(0,orgtbl); string theformula = "@>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f\n@>$2='(format \"%s_%f\" (downcase @1$2) @>>>$1)\n@>$1='(min @4$2 (max @3 @5))\n@>$3=@4$1-((@4$3 / @4$4) + 0.5)\n@1$2='(concat \"2_\" @1$2)\n@1$3 = '(abs @4$4) + '(org-sbe \"x\")\n$4=($1*$2);%.2f\n@1='(concat \"[\" @1 \"]\")\n@3$3='(random 100)\n@4$3='(mod $< 10)"; //string e = theformula; // we need 9.0846 from the above string[] xprs = theformula.split("\n"); int ii = 0; int oo = 0; int r = 0; int c = 0; string fm = ""; foreach (string e in xprs) { if (spew) { print("reading formula : %s\n",e); } string[] ep = e.split("="); ii = -1; oo = -1; r = -1; c = -1; fm = ""; if (ep.length == 2) { ep[0] = ep[0].strip(); ep[1] = ep[1].strip(); if (ep[0] != "" && ep[1] != "") { // TODO: // handle whole row/col target loops, eg: $3=($1*$2) -> for i in rows { cell[i,2] = cell[i,0] * cell[i,1] } if (spew) { print("\tget target cell...\n"); } ii = ep[0].index_of("@"); oo = ep[0].index_of("$"); print("index_of @ is %d, index_of $ is %d\n",ii,oo); if (ii > -1) { string rs = ep[0].substring((ii+1)); r = getrefindex(2,dat.length[0],rs,dat); if (spew) { print("\ttarget row: %d (%s)\n",r,rs); } } if (oo > -1) { string cs = ep[0].substring((oo + 1)); c = getrefindex(2,dat.length[1],cs,dat); if (spew) { print("\ttarget col: %d, (%s)\n",c,cs); } } // target is valid if ((r + c) != -2) { // eval a row loop if (c == -1) { print("\tlooping over columns...\n"); for (int i = 0; i < dat.length[1]; i++) { string ie = ep[1]; if (i > 100) { break; } c = i; if (dat[r,c].has_prefix("--") == false) { ie = doelisp(2,r,c,ie,dat); if (ie.contains("@") || ie.contains("$")) { if (ie[0] != '(') { ie = "(%s)".printf(ie); } } ie = domaths(2,r,c,ie,dat); if ( ie.strip() != "") { if (ie.contains(";")) {fm = doformat(0,ie);} else {fm = ie;} if (spew) { print("\tformula changed dat[%d,%d] from \"%s\" to %s\n\n",r,c,dat[r,c],fm); } dat[r,c] = fm; } } } } else { // eval a column loop if (r == -1) { print("\tlooping over rows...\n"); for (int i = 0; i < dat.length[0]; i++) { string ie = ep[1]; if (i > 100) { break; } r = i; if (dat[r,c].has_prefix("--") == false) { ie = doelisp(2,r,c,ie,dat); if (ie.contains("@") || ie.contains("$")) { if (ie[0] != '(') { ie = "(%s)".printf(ie); } } ie = domaths(2,r,c,ie,dat); if ( ie.strip() != "") { if (ie.contains(";")) {fm = doformat(0,ie);} else {fm = ie;} if (spew) { print("\tformula changed dat[%d,%d] from \"%s\" to %s\n\n",r,c,dat[r,c],fm); } dat[r,c] = fm; } } } } else { // eval once for a cell... ep[1] = doelisp(2,r,c,ep[1],dat); if (ep[1].contains("@") || ep[1].contains("$")) { if (ep[1][0] != '(') { ep[1] = "(%s)".printf(ep[1]); } } ep[1] = domaths(2,r,c,ep[1],dat); if ( ep[1].strip() != "") { if (ep[1].contains(";")) {fm = doformat(0,ep[1]);} else {fm = ep[1];} if (spew) { print("\tformula changed dat[%d,%d] from \"%s\" to %s\n\n",r,c,dat[r,c],fm); } dat[r,c] = fm; } } } if (spew) { print("\n%s\n",reorgtable(0,dat)); } if (spew) { print("\n#+TBLFM: %s\n",theformula); } int64 ofmte = GLib.get_real_time(); if (spew) { print("\ntable formula took %f microts\n\n",(((double) (ofmte - ofmts))/1000000.0) ); } } } } } }
#+RESULTS:
orgtodat took 37.000000 microseconds reading formula : @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is >$4 get prev ref (>)... getrefindex: zero-based cell ref is 6 getrefindex: took 7.000000 microseconds target row: 6 (>$4) getrefindex: input is 4 getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds target col: 3, (4) tblfm checking expression: ((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f tblfm: inner expression: (@1$4..@>>>$4) dosum: (@1$4..@>>>$4) getrefindex: input is 1$4 getrefindex: zero-based cell ref is 0 getrefindex: took 4.000000 microseconds getrefindex: input is 4 getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds dosum: part0 row = 0, col = 3 getrefindex: input is >>>$4. get prev ref (>)... getrefindex: zero-based cell ref is 3 getrefindex: took 4.000000 microseconds getrefindex: input is 4. getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds dosum: part1 row = 3, col = 3 dosum: zero-based cell refs... row1=0, col1=3 row2=3, col2=3 vsum = 328.590000 dosum took 30.000000 microseconds tblfm: sum = 328.590000 tblfm: spliced expression = ((328.590000 / 1000.0 ) * 20.0);%.2f tblfm: inner expression: (328.590000 / 1000.0 ) domaths input .....: 328.590000 / 1000.0 domaths expression : 328.590000 / 1000.0 domaths splice ....: 0.328590 domaths took 9.000000 microseconds tblfm: result = 0.328590 tblfm: spliced expression = (0.328590 * 20.0);%.2f tblfm: inner expression: (0.328590 * 20.0) domaths input .....: 0.328590 * 20.0 domaths expression : 0.328590 * 20.0 domaths splice ....: 6.571800 domaths took 5.000000 microseconds tblfm: result = 6.571800 tblfm: spliced expression = 6.571800;%.2f doformat: input is 6.571800;%.2f doformat took 2.000000 microseconds formula changed dat[6,3] from "" to 6.57 reorgtable took 33.000000 microseconds | AA | BB | CC | DD | |--------+-------+--------+--------| | 68.0 | 39.47 | 128.15 | 337.59 | | 403.88 | 16.21 | 117.03 | -9.0 | | 5.8 | 14.73 | 58.1 | 107.64 | |--------+-------+--------+--------| | | | | 6.57 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.000192 microts reading formula : @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is >$2 get prev ref (>)... getrefindex: zero-based cell ref is 6 getrefindex: took 4.000000 microseconds target row: 6 (>$2) getrefindex: input is 2 getrefindex: zero-based cell ref is 1 getrefindex: took 3.000000 microseconds target col: 1, (2) elisp: lisp expression is '(format "%s_%f" (downcase @1$2) @>>>$1) elisp: spliced comma: '(format "%s_%f" (downcase @1$2) @>>>$1) elisp: outer lisp expression is (format "%s_%f" (downcase @1$2) @>>>$1) elisp inner: iteration 0 replacerefs: input is (downcase @1$2) getrefindex: input is 1$2. getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds getrefindex: input is 2. getrefindex: zero-based cell ref is 1 getrefindex: took 2.000000 microseconds r = 0, c = 1 replacerefs: spliced expression: (downcase BB) replacerefs took 14.000000 microseconds elisp inner: lisp expression: (downcase BB) evallisp: input is (downcase BB) downcase... evallisp downcase took 8.000000 microseconds elisp inner: spliced expression = (format %s_%f bb @>>>$1) elisp inner: iteration 1 replacerefs: input is (format %s_%f bb @>>>$1) getrefindex: input is >>>$1. get prev ref (>)... getrefindex: zero-based cell ref is 3 getrefindex: took 5.000000 microseconds getrefindex: input is 1. getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 3, c = 0 replacerefs: spliced expression: (format %s_%f bb 403.88) replacerefs took 16.000000 microseconds elisp inner: lisp expression: (format %s_%f bb 403.88) evallisp: input is (format %s_%f bb 403.88) format... getting tokens in %s_%f spliced format: bb_%%f spliced format: bb_403.88 evallisp format took 10.000000 microseconds elisp inner: spliced expression = bb_403.88 tblfm checking expression: bb_403.88 formula changed dat[6,1] from "" to bb_403.88 reorgtable took 31.000000 microseconds | AA | BB | CC | DD | |--------+------------+--------+--------| | 68.0 | 39.47 | 128.15 | 337.59 | | 403.88 | 16.21 | 117.03 | -9.0 | | 5.8 | 14.73 | 58.1 | 107.64 | |--------+------------+--------+--------| | | bb_403.88 | | 6.57 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.000311 microts reading formula : @>$1='(min @4$2 (max @3 @5)) get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is >$1 get prev ref (>)... getrefindex: zero-based cell ref is 6 getrefindex: took 3.000000 microseconds target row: 6 (>$1) getrefindex: input is 1 getrefindex: zero-based cell ref is 0 getrefindex: took 2.000000 microseconds target col: 0, (1) elisp: lisp expression is '(min @4$2 (max @3 @5)) elisp: spliced comma: '(min @4$2 (max @3 @5)) elisp: outer lisp expression is (min @4$2 (max @3 @5)) elisp inner: iteration 0 replacerefs: input is (max @3 @5) getrefindex: input is 3.@5. getrefindex: zero-based cell ref is 2 getrefindex: took 2.000000 microseconds r = 2, c = 0 replacerefs: spliced expression: (max 68.0 @5) getrefindex: input is 5. getrefindex: zero-based cell ref is 4 getrefindex: took 2.000000 microseconds r = 4, c = 0 replacerefs: spliced expression: (max 68.0 5.8) replacerefs took 12.000000 microseconds elisp inner: lisp expression: (max 68.0 5.8) evallisp: input is (max 68.0 5.8) max... evallisp took 6.000000 microseconds elisp inner: spliced expression = (min @4$2 68.000000) elisp inner: iteration 1 replacerefs: input is (min @4$2 68.000000) getrefindex: input is 4$2.68.000000. getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds getrefindex: input is 2.68.000000. getrefindex: zero-based cell ref is 1 getrefindex: took 3.000000 microseconds r = 3, c = 1 replacerefs: spliced expression: (min 16.21 68.000000) replacerefs took 12.000000 microseconds elisp inner: lisp expression: (min 16.21 68.000000) evallisp: input is (min 16.21 68.000000) min... evallisp took 6.000000 microseconds elisp inner: spliced expression = 16.210000 tblfm checking expression: 16.210000 formula changed dat[6,0] from "" to 16.210000 reorgtable took 30.000000 microseconds | AA | BB | CC | DD | |------------+------------+--------+--------| | 68.0 | 39.47 | 128.15 | 337.59 | | 403.88 | 16.21 | 117.03 | -9.0 | | 5.8 | 14.73 | 58.1 | 107.64 | |------------+------------+--------+--------| | 16.210000 | bb_403.88 | | 6.57 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.000411 microts reading formula : @>$3=@4$1-((@4$3 / @4$4) + 0.5) get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is >$3 get prev ref (>)... getrefindex: zero-based cell ref is 6 getrefindex: took 3.000000 microseconds target row: 6 (>$3) getrefindex: input is 3 getrefindex: zero-based cell ref is 2 getrefindex: took 3.000000 microseconds target col: 2, (3) tblfm checking expression: (@4$1-((@4$3 / @4$4) + 0.5)) tblfm: inner expression: (@4$3 / @4$4) domaths input .....: @4$3 / @4$4 replacerefs: input is @4$3 / @4$4 getrefindex: input is 4$3...@4$4 getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds getrefindex: input is 3...@4$4 getrefindex: zero-based cell ref is 2 getrefindex: took 3.000000 microseconds r = 3, c = 2 replacerefs: spliced expression: 117.03 / @4$4 getrefindex: input is 4$4 getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds getrefindex: input is 4 getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds r = 3, c = 3 replacerefs: spliced expression: 117.03 / -9.04 replacerefs took 22.000000 microseconds subminus took 1.000000 microseconds domaths expression : 117.03 / -9.04 domaths splice ....: -12.945796 domaths took 33.000000 microseconds tblfm: result = -12.945796 tblfm: spliced expression = (@4$1-(-12.945796 + 0.5)) tblfm: inner expression: (-12.945796 + 0.5) domaths input .....: -12.945796 + 0.5 subminus took 1.000000 microseconds domaths expression : -12.945796 + 0.5 domaths splice ....: -12.445796 domaths took 9.000000 microseconds tblfm: result = -12.445796 tblfm: spliced expression = (@4$1- -12.445796) tblfm: inner expression: (@4$1- -12.445796) domaths input .....: @4$1- -12.445796 replacerefs: input is @4$1- -12.445796 getrefindex: input is 4$1...12.445796 getrefindex: zero-based cell ref is 3 getrefindex: took 4.000000 microseconds getrefindex: input is 1...12.445796 getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 3, c = 0 replacerefs: spliced expression: 403.88- -12.445796 replacerefs took 13.000000 microseconds subminus took 1.000000 microseconds domaths expression : 403.88! -12.445796 domaths splice ....: 416.325796 domaths took 23.000000 microseconds tblfm: result = 416.325796 tblfm: spliced expression = 416.325796 formula changed dat[6,2] from "" to 416.325796 reorgtable took 30.000000 microseconds | AA | BB | CC | DD | |------------+------------+------------+--------| | 68.0 | 39.47 | 128.15 | 337.59 | | 403.88 | 16.21 | 117.03 | -9.0 | | 5.8 | 14.73 | 58.1 | 107.64 | |------------+------------+------------+--------| | 16.210000 | bb_403.88 | 416.325796 | 6.57 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.000546 microts reading formula : @1$2='(concat "2_" @1$2) get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is 1$2 getrefindex: zero-based cell ref is 0 getrefindex: took 4.000000 microseconds target row: 0 (1$2) getrefindex: input is 2 getrefindex: zero-based cell ref is 1 getrefindex: took 3.000000 microseconds target col: 1, (2) elisp: lisp expression is '(concat "2_" @1$2) elisp: spliced comma: '(concat "2_" @1$2) elisp: outer lisp expression is (concat "2_" @1$2) elisp inner: iteration 0 replacerefs: input is (concat "2_" @1$2) getrefindex: input is 1$2. getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds getrefindex: input is 2. getrefindex: zero-based cell ref is 1 getrefindex: took 5.000000 microseconds r = 0, c = 1 replacerefs: spliced expression: (concat "2_" BB) replacerefs took 17.000000 microseconds elisp inner: lisp expression: (concat "2_" BB) evallisp: input is (concat "2_" BB) concat... evallisp concat took 8.000000 microseconds elisp inner: spliced expression = 2_BB tblfm checking expression: 2_BB formula changed dat[0,1] from "BB" to 2_BB reorgtable took 36.000000 microseconds | AA | 2_BB | CC | DD | |------------+------------+------------+--------| | 68.0 | 39.47 | 128.15 | 337.59 | | 403.88 | 16.21 | 117.03 | -9.0 | | 5.8 | 14.73 | 58.1 | 107.64 | |------------+------------+------------+--------| | 16.210000 | bb_403.88 | 416.325796 | 6.57 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.000648 microts reading formula : @1$3 = '(abs @4$4) + '(org-sbe "x") get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is 1$3 getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds target row: 0 (1$3) getrefindex: input is 3 getrefindex: zero-based cell ref is 2 getrefindex: took 3.000000 microseconds target col: 2, (3) elisp: lisp expression is '(abs @4$4) + '(org-sbe "x") elisp: spliced comma: '(abs @4$4) + '(org-sbe "x") elisp: outer lisp expression is (abs @4$4) elisp inner: iteration 0 replacerefs: input is (abs @4$4) getrefindex: input is 4$4. getrefindex: zero-based cell ref is 3 getrefindex: took 4.000000 microseconds getrefindex: input is 4. getrefindex: zero-based cell ref is 3 getrefindex: took 4.000000 microseconds r = 3, c = 3 replacerefs: spliced expression: (abs -9.0) replacerefs took 16.000000 microseconds elisp inner: lisp expression: (abs -9.0) evallisp: input is (abs -9.0) abs... evallisp abs took 6.000000 microseconds elisp inner: spliced expression = 9.000000 elisp: lisp expression is '(abs @4$4) + '(org-sbe "x") elisp: spliced comma: '(abs @4$4) + '(org-sbe "x") elisp: outer lisp expression is (abs @4$4) elisp inner: iteration 0 replacerefs: input is (abs @4$4) getrefindex: input is 4$4. getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds getrefindex: input is 4. getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds r = 3, c = 3 replacerefs: spliced expression: (abs -9.0) replacerefs took 14.000000 microseconds elisp inner: lisp expression: (abs -9.0) evallisp: input is (abs -9.0) abs... evallisp abs took 5.000000 microseconds elisp inner: spliced expression = 9.000000 elisp: lisp expression is '(abs @4$4) + '(org-sbe "x") elisp: spliced comma: '(abs @4$4) + '(org-sbe "x") elisp: outer lisp expression is (abs @4$4) elisp inner: iteration 0 replacerefs: input is (abs @4$4) getrefindex: input is 4$4. getrefindex: zero-based cell ref is 3 getrefindex: took 5.000000 microseconds getrefindex: input is 4. getrefindex: zero-based cell ref is 3 getrefindex: took 2.000000 microseconds r = 3, c = 3 replacerefs: spliced expression: (abs -9.0) replacerefs took 13.000000 microseconds elisp inner: lisp expression: (abs -9.0) evallisp: input is (abs -9.0) abs... evallisp abs took 4.000000 microseconds elisp inner: spliced expression = 9.000000 tblfm checking expression: 9.000000(org-sbe "x") tblfm: inner expression: (org-sbe "x") domaths input .....: org-sbe "x" subminus took 1.000000 microseconds domaths expression : org -sbe "x" domaths took 5.000000 microseconds tblfm: result = org -sbe "x" tblfm: spliced expression = 9.000000org -sbe "x" formula changed dat[0,2] from "CC" to 9.000000org -sbe "x" reorgtable took 31.000000 microseconds | AA | 2_BB | 9.000000org -sbe "x" | DD | |------------+------------+-----------------------+--------| | 68.0 | 39.47 | 128.15 | 337.59 | | 403.88 | 16.21 | 117.03 | -9.0 | | 5.8 | 14.73 | 58.1 | 107.64 | |------------+------------+-----------------------+--------| | 16.210000 | bb_403.88 | 416.325796 | 6.57 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.000806 microts reading formula : $4=($1*$2);%.2f get target cell... index_of @ is -1, index_of $ is 0 getrefindex: input is 4 getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds target col: 3, (4) looping over rows... tblfm checking expression: ($1*$2);%.2f tblfm: inner expression: ($1*$2) domaths input .....: $1*$2 replacerefs: input is $1*$2 getrefindex: input is 1.$2 getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 0, c = 0 replacerefs: spliced expression: AA*$2 getrefindex: input is 2 getrefindex: zero-based cell ref is 1 getrefindex: took 3.000000 microseconds r = 0, c = 1 replacerefs: spliced expression: AA* 2_BB2 replacerefs took 14.000000 microseconds domaths expression : AA* 2_BB2 ERROR: __ or _2___2 are not float domaths took 21.000000 microseconds tblfm: result = AA* 2_BB2 tblfm: spliced expression = AA* 2_BB2;%.2f doformat: input is AA* 2_BB2;%.2f doformat took 2.000000 microseconds formula changed dat[0,3] from "DD" to 0.00 tblfm checking expression: ($1*$2);%.2f tblfm: inner expression: ($1*$2) domaths input .....: $1*$2 replacerefs: input is $1*$2 getrefindex: input is 1.$2 getrefindex: zero-based cell ref is 0 getrefindex: took 2.000000 microseconds r = 2, c = 0 replacerefs: spliced expression: 68.0*$2 getrefindex: input is 2 getrefindex: zero-based cell ref is 1 getrefindex: took 2.000000 microseconds r = 2, c = 1 replacerefs: spliced expression: 68.0*39.472 replacerefs took 15.000000 microseconds domaths expression : 68.0*39.472 domaths splice ....: 2684.096000 domaths took 22.000000 microseconds tblfm: result = 2684.096000 tblfm: spliced expression = 2684.096000;%.2f doformat: input is 2684.096000;%.2f doformat took 2.000000 microseconds formula changed dat[2,3] from "337.59" to 2684.10 tblfm checking expression: ($1*$2);%.2f tblfm: inner expression: ($1*$2) domaths input .....: $1*$2 replacerefs: input is $1*$2 getrefindex: input is 1.$2 getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 3, c = 0 replacerefs: spliced expression: 403.88*$2 getrefindex: input is 2 getrefindex: zero-based cell ref is 1 getrefindex: took 3.000000 microseconds r = 3, c = 1 replacerefs: spliced expression: 403.88*16.212 replacerefs took 14.000000 microseconds domaths expression : 403.88*16.212 domaths splice ....: 6547.702560 domaths took 20.000000 microseconds tblfm: result = 6547.702560 tblfm: spliced expression = 6547.702560;%.2f doformat: input is 6547.702560;%.2f doformat took 2.000000 microseconds formula changed dat[3,3] from "-9.0" to 6547.70 tblfm checking expression: ($1*$2);%.2f tblfm: inner expression: ($1*$2) domaths input .....: $1*$2 replacerefs: input is $1*$2 getrefindex: input is 1.$2 getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 4, c = 0 replacerefs: spliced expression: 5.8*$2 getrefindex: input is 2 getrefindex: zero-based cell ref is 1 getrefindex: took 2.000000 microseconds r = 4, c = 1 replacerefs: spliced expression: 5.8*14.732 replacerefs took 13.000000 microseconds domaths expression : 5.8*14.732 domaths splice ....: 85.445600 domaths took 19.000000 microseconds tblfm: result = 85.445600 tblfm: spliced expression = 85.445600;%.2f doformat: input is 85.445600;%.2f doformat took 2.000000 microseconds formula changed dat[4,3] from "107.64" to 85.45 tblfm checking expression: ($1*$2);%.2f tblfm: inner expression: ($1*$2) domaths input .....: $1*$2 replacerefs: input is $1*$2 getrefindex: input is 1.$2 getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 6, c = 0 replacerefs: spliced expression: 16.210000*$2 getrefindex: input is 2 getrefindex: zero-based cell ref is 1 getrefindex: took 2.000000 microseconds r = 6, c = 1 replacerefs: spliced expression: 16.210000* bb_403.882 replacerefs took 13.000000 microseconds domaths expression : 16.210000* bb_403.882 domaths splice ....: 6546.927220 domaths took 20.000000 microseconds tblfm: result = 6546.927220 tblfm: spliced expression = 6546.927220;%.2f doformat: input is 6546.927220;%.2f doformat took 2.000000 microseconds formula changed dat[6,3] from "6.57" to 6546.93 reorgtable took 31.000000 microseconds | AA | 2_BB | 9.000000org -sbe "x" | 0.00 | |------------+------------+-----------------------+---------| | 68.0 | 39.47 | 128.15 | 2684.10 | | 403.88 | 16.21 | 117.03 | 6547.70 | | 5.8 | 14.73 | 58.1 | 85.45 | |------------+------------+-----------------------+---------| | 16.210000 | bb_403.88 | 416.325796 | 6546.93 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.001008 microts reading formula : @1='(concat "[" @1 "]") get target cell... index_of @ is 0, index_of $ is -1 getrefindex: input is 1 getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds target row: 0 (1) looping over columns... elisp: lisp expression is '(concat "[" @1 "]") elisp: spliced comma: '(concat "[" @1 "]") elisp: outer lisp expression is (concat "[" @1 "]") elisp inner: iteration 0 replacerefs: input is (concat "[" @1 "]") getrefindex: input is 1..... getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 0, c = 0 replacerefs: spliced expression: (concat "[" AA "]") replacerefs took 9.000000 microseconds elisp inner: lisp expression: (concat "[" AA "]") evallisp: input is (concat "[" AA "]") concat... evallisp concat took 8.000000 microseconds elisp inner: spliced expression = [AA] tblfm checking expression: [AA] formula changed dat[0,0] from "AA" to [AA] elisp: lisp expression is '(concat "[" @1 "]") elisp: spliced comma: '(concat "[" @1 "]") elisp: outer lisp expression is (concat "[" @1 "]") elisp inner: iteration 0 replacerefs: input is (concat "[" @1 "]") getrefindex: input is 1..... getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 0, c = 1 replacerefs: spliced expression: (concat "[" 2_BB "]") replacerefs took 10.000000 microseconds elisp inner: lisp expression: (concat "[" 2_BB "]") evallisp: input is (concat "[" 2_BB "]") concat... evallisp concat took 7.000000 microseconds elisp inner: spliced expression = [2_BB] tblfm checking expression: [2_BB] formula changed dat[0,1] from " 2_BB" to [2_BB] elisp: lisp expression is '(concat "[" @1 "]") elisp: spliced comma: '(concat "[" @1 "]") elisp: outer lisp expression is (concat "[" @1 "]") elisp inner: iteration 0 replacerefs: input is (concat "[" @1 "]") getrefindex: input is 1..... getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 0, c = 2 replacerefs: spliced expression: (concat "[" 9.000000org -sbe "x" "]") replacerefs took 9.000000 microseconds elisp inner: lisp expression: (concat "[" 9.000000org -sbe "x" "]") evallisp: input is (concat "[" 9.000000org -sbe "x" "]") concat... evallisp concat took 9.000000 microseconds elisp inner: spliced expression = [9.000000org-sbex] tblfm checking expression: [9.000000org-sbex] formula changed dat[0,2] from " 9.000000org -sbe "x"" to [9.000000org-sbex] elisp: lisp expression is '(concat "[" @1 "]") elisp: spliced comma: '(concat "[" @1 "]") elisp: outer lisp expression is (concat "[" @1 "]") elisp inner: iteration 0 replacerefs: input is (concat "[" @1 "]") getrefindex: input is 1..... getrefindex: zero-based cell ref is 0 getrefindex: took 3.000000 microseconds r = 0, c = 3 replacerefs: spliced expression: (concat "[" 0.00 "]") replacerefs took 10.000000 microseconds elisp inner: lisp expression: (concat "[" 0.00 "]") evallisp: input is (concat "[" 0.00 "]") concat... evallisp concat took 7.000000 microseconds elisp inner: spliced expression = [0.00] tblfm checking expression: [0.00] formula changed dat[0,3] from "0.00" to [0.00] reorgtable took 32.000000 microseconds | [AA] | [2_BB] | [9.000000org-sbex] | [0.00] | |------------+------------+---------------------+---------| | 68.0 | 39.47 | 128.15 | 2684.10 | | 403.88 | 16.21 | 117.03 | 6547.70 | | 5.8 | 14.73 | 58.1 | 85.45 | |------------+------------+---------------------+---------| | 16.210000 | bb_403.88 | 416.325796 | 6546.93 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.001170 microts reading formula : @3$3='(random 100) get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is 3$3 getrefindex: zero-based cell ref is 2 getrefindex: took 3.000000 microseconds target row: 2 (3$3) getrefindex: input is 3 getrefindex: zero-based cell ref is 2 getrefindex: took 3.000000 microseconds target col: 2, (3) elisp: lisp expression is '(random 100) elisp: spliced comma: '(random 100) elisp: outer lisp expression is (random 100) elisp inner: iteration 0 elisp inner: lisp expression: (random 100) evallisp: input is (random 100) rand... evallisp random took 39.000000 microseconds elisp inner: spliced expression = 72 tblfm checking expression: 72 formula changed dat[2,2] from "128.15" to 72 reorgtable took 35.000000 microseconds | [AA] | [2_BB] | [9.000000org-sbex] | [0.00] | |------------+------------+---------------------+---------| | 68.0 | 39.47 | 72 | 2684.10 | | 403.88 | 16.21 | 117.03 | 6547.70 | | 5.8 | 14.73 | 58.1 | 85.45 | |------------+------------+---------------------+---------| | 16.210000 | bb_403.88 | 416.325796 | 6546.93 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.001281 microts reading formula : @4$3='(mod $< 10) get target cell... index_of @ is 0, index_of $ is 2 getrefindex: input is 4$3 getrefindex: zero-based cell ref is 3 getrefindex: took 3.000000 microseconds target row: 3 (4$3) getrefindex: input is 3 getrefindex: zero-based cell ref is 2 getrefindex: took 3.000000 microseconds target col: 2, (3) elisp: lisp expression is '(mod $< 10) elisp: spliced comma: '(mod $< 10) elisp: outer lisp expression is (mod $< 10) elisp inner: iteration 0 replacerefs: input is (mod $< 10) getrefindex: input is <.10. get next ref (<)... getrefindex: zero-based cell ref is 3 getrefindex: took 5.000000 microseconds r = 3, c = 3 replacerefs: spliced expression: (mod 6547.70 10) replacerefs took 11.000000 microseconds elisp inner: lisp expression: (mod 6547.70 10) evallisp: input is (mod 6547.70 10) mod... fmod(6547.700000,10.000000) evallisp fmod took 10.000000 microseconds elisp inner: spliced expression = 7.700000 tblfm checking expression: 7.700000 formula changed dat[3,2] from "117.03" to 7.700000 reorgtable took 33.000000 microseconds | [AA] | [2_BB] | [9.000000org-sbex] | [0.00] | |------------+------------+---------------------+---------| | 68.0 | 39.47 | 72 | 2684.10 | | 403.88 | 16.21 | 7.700000 | 6547.70 | | 5.8 | 14.73 | 58.1 | 85.45 | |------------+------------+---------------------+---------| | 16.210000 | bb_403.88 | 416.325796 | 6546.93 | ,#+TBLFM: @>$4=((vsum(@1$4..@>>>$4) / 1000.0 ) * 20.0);%.2f @>$2='(format "%s_%f" (downcase @1$2) @>>>$1) @>$1='(min @4$2 (max @3 @5)) @>$3=@4$1-((@4$3 / @4$4) + 0.5) @1$2='(concat "2_" @1$2) @1$3 = '(abs @4$4) + '(org-sbe "x") $4=($1*$2);%.2f @1='(concat "[" @1 "]") @3$3='(random 100) @4$3='(mod $< 10) table formula took 0.001370 microts
TIME:cmd:

GNU command to quickly check max physical ram usage of a command in kilobytes:

/usr/bin/time -f "%M" ./r3 mytestscript.r3 mytestargfile.org
TTYPLOT:cmd:
Plot memory usage of a process with ttyplot,
ttyplot by Antoni Sawicki is here

Virtual memory:

FRWN=$(pidof "frownedupon") && while :; do pmap -x $FRWN | grep -oP '^total kB\s+\K\d+' | numfmt --from-unit K --to-unit M; sleep 5; done | ./ttyplot -u M -c "█"
#+RESULTS:
.: ttyplot :. ^ 1163.0 M █ ██ │ ███████████████████████████████ │ ███████████████████████████████ │ 872.2 M ████████████████████████████████ │ ██████████████████████████████████ │ ██████████████████████████████████ │ ██████████████████████████████████ │ 581.5 M ██████████████████████████████████ │ ██████████████████████████████████ │ ██████████████████████████████████ │ ██████████████████████████████████ │ 290.8 M ██████████████████████████████████ │ ██████████████████████████████████ │ ██████████████████████████████████ │ ██████████████████████████████████ └───────────────────────────────────────────────────────────────────────> █ last=1155.0 min=919.0 max=1163.0 avg=1139.8 M i Apr 28 22:07:37 2023 github.com/tenox7/ttyplot 1.4
Physical memory:

FRWN=$(pidof "frownedupon") && while :; do pmap -x $FRWN | grep -oP '^total kB\s+\d+\s+\K\d+' | numfmt --from-unit K --to-unit M; sleep 5; done | ./ttyplot -u M -c "█"
.: ttyplot :. ^ 102.0 M ██████ │ █████████████████ │ █████████████████████████████████████████████ │ 76.5 M █████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ 51.0 M ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ 25.5 M ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ │ ███████████████████████████████████████████████ └───────────────────────────────────────────────────────────────────────> █ last=102.0 min=76.0 max=102.0 avg=95.3 M Fri Apr 28 22:24:43 2023 github.com/tenox7/ttyplot 1.4
ORGNQQ:rebol3:

screenie
gifded queue to export an orgfile to html.
Used to make this site.

Mostly written in Rebol3, a beutiful low-effort hight-payoff language... reminds me of Python before the crowds...

The queue:
preflight
shellscript node to clear out pub dirs
loader
hardcoded vala node to load a file
org title:rebol3:


org headlines:rebol3:


org noexport:wip::rebol3:
needs to be fixed as the technique doesn't cover all possible cases


org tables:rebol3:


org source:rebol3:


org checkboxes:rebol3:


org lists:rebol3:


org properties:rebol3:


org dates:rebol3:


org links:rebol3:


org style:rebol3:


org include:rebol3:


linebreak content:wip::rebol3:
fix to ignore tags in xmp or pre


insert script
html node that simply replaces <!--[lastres]--> with the incoming text
prepend css
html node that simply replaces <!--[lastres]--> with the incoming text
save fonts:rebol3:


save html
Hardcoded Vala node that saves the incoming text to index file
publish
shellscript node to copy everything to a publish location
ORG SBE:org:
REBOL[] r: read/lines https://www.rba.gov.au/rss/rss-cb-exchange-rates.xml b: copy "" foreach l r [ if parse l [ thru ">AU:" a: to "EUR" :a copy b to "EUR" to end ] [ break ]] probe (1.0 / (to-decimal trim b)) #+NAME: x
#+RESULTS:
: 1.62760416666667

| ITEM  |  VAL |  XCH | ATAX |   AUD |
|-------+------+------+------+-------|
| AUD   | 88.0 |  1.0 | 0.76 |  66.9 |
| EUR   | 60.0 | 1.63 | 0.52 |  50.9 |
|-------+------+------+------+-------|
| TOTAL |      |      |      | 117.8 |
#+TBLFM: @3$3='(org-sbe "x");%.2f :: $5=(($2*$3)*$4);%.1f :: @>$5=vsum(@I$5..@II$5);%.1f

MONO:fluff:
SPACE
|---+-----------------+-------+--------+---------------------------+-------+--------+-------------------------------------------|
| ! | font            | serif | width  | code                      | table | plot   | subjective note                           |
|---+-----------------+-------+--------+---------------------------+-------+--------+-------------------------------------------|
| * | FiraCode        | no    | med    | makes assumptions         | OK    | OK     | lighter, more readable alternative to APL |
|   | FreeMono        | yes   | med    | heavy semicolon, quotes   | OK    | OK     | clean serif, safe choice, bit light tho   |
|   | APL385          | no    | med    | OK                        | OK    | OK     | very clean, too spread out for prose      |
|   | IBMPlexMono     | no    | med    | OK                        | OK    | OK     | clean unremarkable safe choice            |
|   | JetBrainsMono   | no    | med    | OK                        | OK    | OK     | unremarkable safe choice                  |
|   | TerminusTTF     | no    | med    | OK                        | OK    | OK     | g and y are crushed, too blocky for prose |
|   | Monoid          | no    | narrow | OK                        | OK    | WONKY  | clean, need to scale it down              |
|   | AnkaCoder       | no    | med    | strange zero              | OK    | WONKY  | bit too busy                              |
|   | F25BankPrinter  | no    | wide   | OK                        | WONKY | WONKY  | heavy, too wide                           |
|   | LibertinusMono  | yes   | wide   | OK                        | OK    | WONKY  | feels scaled in x                         |
|   | SVBasicManual   | no    | med    | no curly brace, open zero | GREAT | BUSTED | close to mono eurostile, worth fixing!    |
|   | VerilySerifMono | yes   | med    | OK                        | OK    | BUSTED | bit too busy                              |
|   | crystal         | no    | narrow | OK                        | OK    | BUSTED | clean, not great for prose                |
|   | Courier Prime   | no    | wide   | OK                        | OK    | BUSTED | clean, not great for prose                |
|   | SomeType Mono   | no    | wide   | OK                        | OK    | BUSTED | clean, better for prose than the above    |
| * | LuxiMono        | yes   | wide   | OK                        | OK    | BUSTED | stodge unremarkable mono serif            |
|   | AAActualMono    | no    | narrow | OK                        | OK    | BUSTED | yeah not actual mono...                   |
|   | SelectricMono   | yes   | med    | fucky quotes              | WONKY | BUSTED | letters too spread-out                    |
|   | TeXGyreCursor   | yes   | med    | OK                        | OK    | BUSTED | clean light serif                         |
|   | NotCourierSans  | no    | wide   | weird semicolon           | OK    | BUSTED | clean wide font                           |
|   | CutiveMono      | yes   | wide   | OK                        | OK    | BUSTED | too spread-out, very light                |
|---+-----------------+-------+--------+---------------------------+-------+--------+-------------------------------------------|

THE TAB KEY:el:
From .emacs
(setq-default indent-tabs-mode 'only) (setq-default tab-width 4) (setq-default tab-stop-list (number-sequence 4 120 4)) (electric-indent-mode -1) (add-hook 'after-change-major-mode-hook (lambda() (electric-indent-mode -1))) (global-set-key (kbd "<backspace>") 'backward-delete-char) (defun cpb-editing-hook () ;; tab indent unindent ;; posted by Stanley Bak ;; edited by 'mythicalcoder' ;; at: https://stackoverflow.com/questions/2249955/emacs-shift-tab-to-left-shift-the-block (defun just-insert-tab () "Insert a tab char. (ASCII 9, \t)" (interactive) (insert "\t") ) (defun just-insert-newline () "Insert a newline char" (interactive) (insert "\n") ) (defun indent-region-custom(numSpaces) (if (use-region-p) (progn ; default to start and end of current line (setq regionStart (line-beginning-position)) (setq regionEnd (line-end-position)) ; if there's a selection, use that instead of the current line (when (use-region-p) (setq regionStart (region-beginning)) (setq regionEnd (region-end)) ) (save-excursion ; restore the position afterwards (goto-char regionStart) ; go to the start of region (setq start (line-beginning-position)) ; save the start of the line (goto-char regionEnd) ; go to the end of region (setq end (line-end-position)) ; save the end of the line (indent-rigidly start end numSpaces) ; indent between start and end (setq deactivate-mark nil) ; restore the selected region ) ) (just-insert-tab) ) ) (defun untab-region (N) (interactive "p") (indent-region-custom -4) ) (defun tab-region (N) (interactive "p") (indent-region-custom 4) ) (setq indent-tabs-mode 'only) (setq tab-width 4) (setq tab-stop-list (number-sequence 4 120 4)) (electric-indent-mode -1) (local-set-key (kbd "<backspace>") 'backward-delete-char) (local-set-key (kbd "<M-S-up>") 'move-line-up) (local-set-key (kbd "<M-S-down>") 'move-line-down) (local-set-key (kbd "<backtab>") 'untab-region) (local-set-key (kbd "<tab>") 'tab-region) (local-set-key (kbd "TAB") 'tab-region) (local-set-key (kbd "<return>") 'just-insert-newline) ;; special cases for modes with a keymap (define-key emacs-lisp-mode-map (kbd "<return>") 'just-insert-newline) (define-key emacs-lisp-mode-map (kbd "TAB") 'tab-region) (define-key emacs-lisp-mode-map (kbd "<tab>") 'tab-region) (define-key emacs-lisp-mode-map (kbd "<backtab>") 'untab-region) ) ;; tab = tab for rebol3 (add-hook 'rebol3-mode-hook 'cpb-editing-hook) ;; tab = tab for python (add-hook 'python-mode-hook 'cpb-editing-hook) ;; tab = tab for xml (add-hook 'nxml-mode-hook 'cpb-editing-hook) ;; tab = tab for html (add-hook 'html-mode-hook 'cpb-editing-hook) ;; tab = tab or vala (add-hook 'vala-mode-hook 'cpb-editing-hook) ;; tab = tab for elsip (add-hook 'emacs-lisp-mode-hook 'cpb-editing-hook) ;; tab = tab for shell (add-hook 'shell-mode-hook 'cpb-editing-hook) ;; org src blocks (setq org-src-tab-acts-natively t) (setq org-src-preserve-indentation t) (setq org-edit-src-content-indentation 0) (setq org-src-fontify-natively t)
R3 RENAME:rebol3:
Batch rename with Rebol3.

sample name...: "Within Destruction - Lotus - 11 P.O.P..flac"
desired output: "within_destruction_lotus_11_pop.flac"

the crude way:
;; rename test ;; by c.p.brown 2023 f: read %./*.flac foreach x f [ n: lowercase (to-string x) replace n ".flac" "" n: reword/escape n [ " " "_" "." "" "-" "" "(" "" ")" "" ] none replace/all n "__" "_" n: join n ".flac" call compose [ "cp" (to-string x) (n) ] ]
the more reliable way, no duplicate underscores:
;; rename test using parse ;; by c.p.brown 2023 f: read %./*.flac nup: charset ".-()" foreach x f [ n: lowercase to-string x parse n [ a: any [ [ not ".flac" ] remove nup | skip ] :a any [ remove some #" " insert "_" | skip ] ] print [ "renaming from:" x "^/ to:" n "^/"] t: open/new/write to-file n write/binary t read/binary x ]
another test:
"Within Destruction - Lotus - (.- )- 11 P.O.P..flac"
output:
"within_destruction_lotus_11_pop.flac"
LYNX CRAWL:cmd:
lynx --dump --listonly --nonumbers http://www.yourdomain.com | grep http | grep yourdomain.com | sort | uniq > links.txt
then convert to a xml:
wrap each line in <url><loc> ... <loc><url>
insert a header:

<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
then append </urlset>
and save as sitemap.xml
FIREJAIL FIREFOX:cmd:
sed -i 's/Exec=.*/Exec=firejail firefox -no-remote/g' /usr/share/applications/firefox.desktop
It won't take at first, so run the above command before loading firefox, open and close firefox, run the command again, wait a few seconds, run it a third time... then it should be firejailed. Check by looking at:

file:///home/yourusername
You should only see folders that firefox needs to use.
DUI:gtk3::red:
screenie
Dynamic ui test.
Creates fields/labels from items in a data structure.

Here for reference only. I've since moved from Red to Vala.

Red [ needs 'view ] ;; tag used for single line strings (fields), thanks to Boleslav Březovský re: "string" vs {string} recycle/off articledata: context[ thisis: 'articledata name: <article name> index: 1 ;; used for sorting poster: %./imagefile.png video: %./videofile.mp4 article: {article text^/goes here} onclick: protocol://path.goes.here credits: {credits^/* go^/* here} references: [ protocol://url.goes.here protocol://url.goes.here ] ] getext: func [ f ] [ last split to-string f "." ] makeparams: func [ d ] [ k: d/thisis w: words-of d v: values-of d i: 1 n: copy [] append n [ below ] foreach p w [ switch mold type? v/:i [ "integer!" [ append n compose/deep [ across text (to-string p) text (to-string v/(i)) return below ] ] "tag!" [ append n compose/deep [ text (to-string p) field 200x55 with [ text: (to-string v/(i)) ] on-change [ (to-set-path compose [ (k) (p) ]) to-tag face/text src/text: (quote mold ) (k) ] ] probe n ] "string!" [ append n compose/deep [ text (to-string p) area 200x100 with [ text: (to-string v/(i)) ] on-change [ (to-set-path compose [ (k) (p) ]) face/text src/text: (quote mold ) (k) ] ] ] "url!" [ append n compose/deep [ text (to-string p) field 200x55 with [ text: (to-string v/(i)) ] on-change [ (to-set-path compose [ (k) (p) ]) to-url face/text src/text: (quote mold ) (k) ] ] ] "block!" [ append n compose/deep [ text (to-string p) text-list 200x100 data [(v/(i))] on-change [ (to-set-path compose [ (k) (p) ]) face/data src/text: (quote mold ) (k) ] ] ] "file!" [ either (getext v/:i) = "png" [ append n compose/deep [ text (to-string p) image 200x200 with [ image: load (v/(i)) ] on-up [ img: request-file/filter ["pics" "*.png"] unless none? img [ img: to-file rejoin [ "./" second split-path img ] if exists? img [ face/image: load img unless none? face/image [ (to-set-path compose [ (k) (p) ]) img src/text: (quote mold ) (k) ] ] ] ] ] ] [ print [ "file not a png, skipping..." ] ] ] ] i: i + 1 ] n ] nudgev: func [ ] [ aa/offset/x: 0 pp/offset/y: 0 vv/offset/y: 0 aa/offset/y: 0 bb/offset/y: 0 aa/size/x: vv/offset/x src/size/x: aa/size/x - 20 bb/offset/x: vv/offset/x + vv/size/x bb/size/x: pp/size/x - (vv/offset/x + vv/size/x) cc/size/x: bb/size/x - 70 ] view/tight/flags/options [ title "auto param test" pp: panel 650x600 255.0.0 [ aa: panel 300x600 30.30.30 [ below src: area 300x510 40.40.40 font-name "consolas" font-size 10 font-color 128.255.128 doit: button 120x55 30.30.30 "update" font-name "consolas" font-size 16 font-color 128.128.128 [ tmp: do [ reduce load src/text ] articledata: reduce copy tmp/1 src/text: (mold articledata) clear cc/pane append cc/pane layout/only makeparams articledata ] ] vv: panel 10x600 15.15.15 loose on-drag [ nudgev ] bb: panel 300x600 30.30.30 [ cc: panel 300x1000 60.60.60 loose [ ] on-drag [ face/offset/x: 10 face/offset/y: min face/offset/y 10 ] on-wheel [ either event/picked > 0 [ face/offset/y: face/offset/y + 40 ] [ face/offset/y: face/offset/y - 40 ] bo: (bb/size/y - face/size/y) - 100 bo: min bo 0 face/offset/y: max bo min face/offset/y 10 ] ] ] do [ aa/offset/x: 0 append cc/pane layout/only makeparams articledata src/text: (mold articledata) nudgev ] ] [ resize ] [ actors: object [ on-resizing: function [ face event ] [ if face/size/x > 500 [ if face/size/y > 300 [ pp/size: face/size vv/size/y: face/size/y aa/size/y: face/size/y bb/size/y: face/size/y vv/offset/x: face/size/x - (bb/size/x + vv/size/x) doit/offset/y: face/size/y - (doit/size/y + 100) src/size/y: doit/offset/y - 20 nudgev ] ] ] ] ]
THECOVID:cmd::py:
screenie
Script to watch the 2nd nth wave of covid in NSW 2020.

Linux: Import into org via src block, where:
arg 0 = days to check
arg 1 = column: 0 = distance, 1 = date, 2 = postcode, 3 = burb

#+name=covids #+BEGIN_SRC sh :dir ~/Desktop/cvd/ :session :results raw :results drawer python3 cvd_linux.py 2 1 #+END_SRC :results: 22 new cases | DIST | DATE | POST | SUBURB | |----------+--------+------+------------------------| | 56.82km | 200806 | 2046 | Canada Bay (A) | | 50.67km | 200806 | 2560 | Campbelltown (C) (NSW) | | 36.63km | 200806 | 2176 | Fairfield (C) | | 39.29km | 200806 | 2145 | Cumberland (A) | | 39.29km | 200806 | 2145 | Cumberland (A) | | 36.63km | 200806 | 2176 | Fairfield (C) | | 44.71km | 200806 | 2161 | Cumberland (A) | | 140.87km | 200806 | 2303 | Newcastle (C) | | 45.25km | 200806 | 2150 | Parramatta (C) | | 140.87km | 200806 | 2303 | Newcastle (C) | |----------+--------+------+------------------------| | | | | = 10 | |----------+--------+------+------------------------| | 40.52km | 200805 | 2164 | Fairfield (C) | | 140.87km | 200805 | 2303 | Newcastle (C) | | 62.19km | 200805 | 2042 | Sydney (C) | | 66.29km | 200805 | 2011 | Sydney (C) | | 38.78km | 200805 | 2177 | Fairfield (C) | | 44.33km | 200805 | 2170 | Liverpool (C) | | 43.24km | 200805 | 2166 | Fairfield (C) | | 65.63km | 200805 | 2017 | Sydney (C) | | 44.71km | 200805 | 2161 | Cumberland (A) | | 45.25km | 200805 | 2150 | Parramatta (C) | | 45.25km | 200805 | 2150 | Parramatta (C) | | 44.71km | 200805 | 2161 | Cumberland (A) | |----------+--------+------+------------------------| | | | | = 12 | |----------+--------+------+------------------------| :end:
import urllib.request import datetime import os from math import sin, cos, sqrt, atan2, radians import sys # pythonista widget to show recent covid cases during the 2nd wave... # written by c.p.brown using sources acknowledged below. # last updated August 2020 # covid case data sourced from: # https://data.nsw.gov.au/data/dataset/covid-19-cases-by-location # postcode location dat sourced from Soon Van 'randomecho' : https://gist.github.com/randomecho/5020859 # ... who took and cribbed from blog.datalicious.com/free-download-all-australian-postcodes-geocod (now a 404 site) # so this 3rd(or more)-hand data probably contains errors, use at your own risk # getdist function posted by gwaramadze & fixed by Michael0x2a on stackexchange: https://stackoverflow.com/questions/19412462/getting-distance-between-two-points-based-on-latitude-longitude # who got it from Andrew Hedges: # https://andrew.hedges.name/experiments/haversine/ # who got it from Bob Chamberlain (site unknown) # distances rounded to 0.5 km for table grouping cdr = os.path.split(os.path.realpath('cvd_linux.py'))[0] rcsv = 'https://data.nsw.gov.au/data/dataset/97ea2424-abaf-4f3e-a9f2-b5c883f42b6a/resource/2776dbb8-f807-4fb2-b1ed-184a6fc2c8aa/download/covid-19-cases-by-notification-date-location-and-likely-source-of-infection.csv' pcod = cdr + "/nswpostcodes.csv" mypos = (-33.689990, 150.546212) ago = 2 # number of prior days to check for cases cases = [] def getdist(la, lb, oa, ob) : # function posted by gwaramadze & fixed by Michael0x2a on stackexchange, referring to Andrew Hedges, who referred to Bob Chamberlain # https://andrew.hedges.name/experiments/haversine/ # https://stackoverflow.com/questions/19412462/getting-distance-between-two-points-based-on-latitude-longitude # approximate radius of earth in km R = 6371.0 lat1 = lb lon1 = ob lat2 = la lon2 = oa dlon = radians(lon2 - lon1) dlat = radians(lat2 - lat1) a = (sin(dlat / 2) * sin(dlat / 2) + cos(radians(lat1)) * cos(radians(lat2)) * sin(dlon / 2) * sin(dlon / 2)) c = 2 * atan2(sqrt(a), sqrt(1 - a)) distance = R * c return(distance) def orgtab(s) : # function to clean up a non-aligned org table # intedned for editing orgfile tables in editorial (ios) lines = s.split('\n') mxl = [] tbl = "" # get max column widths for i in lines : p = i.split('|') c = 0 for s in p : if s != "" : sl = len(s) if len(mxl) <= c : mxl.append(sl) else : if sl > mxl[c] : mxl[c] = sl c = c + 1 # have the max column widths, now use them to pad cells... for i in lines : p = i.split('|') c = 0 row = "|" if len(i.strip()) == 0 : row = "" for s in p : if s != "" : q = s.ljust(mxl[c]," ") if s.strip() != "" : if s[0] == "-" : q = s.ljust(mxl[c],"-") if s[0] == "-" : if c < len(p) - 3 : row = row + q + "+" else : row = row + q + "|" else : row = row + q + "|" c = c + 1 if row.strip() != "" : row = row + "\n" tbl = tbl + row lines = 0 return tbl def render(s,u,r) : if len(cases) > 0 : # write case count c = str(len(cases)) + " new cases" # sort and group cases using column key (r) values = set(map(lambda x:x[r], cases)) if r == 1 : values = sorted(values, reverse=True) else : values = sorted(values) cg = [[y for y in cases if y[r]==x] for x in values] # org-style table header o = "| DIST | DATE | POST | SUBURB |\n" o = o + "|------|------|------|--------|\n" for g in cg : gl = len(g) for i in g : o = o + "| " + "{0:.2f}".format(i[0]) + "km | " + i[1] + " | " + i[2] + " | " + i[3] + ' |\n' o = o + "|------|------|------|--------|\n" o = o + "| | | | = " + str(len(g)) + "|\n" o = o + "|------|------|------|--------|\n" # tidy-up the table. # inefficient doing this after the fact, but testing a general-purpose text-editing function for later o = orgtab(o) # update ui return[o,c] else : o = "" c = "no new cases" return[o,c] def btap() : r = int(sys.argv[2]) ago = int(sys.argv[1]) o = "" c = "" # get the covid data from csv ug = urllib.request.urlopen(rcsv) csv = ug.read().splitlines()[1:] ug = 0 # extract date info td = datetime.date.today() tx = td.toordinal() tw = td.strftime('%a') tt = td.day # get postcode locations from csv f = open(pcod,'r') pcsv = f.read().splitlines() f.close() pcc = [] loc = [] for p in pcsv : seg = p.split(';') if not int(seg[0]) in pcc : pcc.append(int(seg[0])) ploc = (float(seg[2]),float(seg[3])) loc.append(ploc) pcsv = 0 latest = [] # gather and format covid cases, filtered by age in days for l in csv : l = l.decode('utf-8') parts = l.split(',') if parts[1].strip() != '' : dt = parts[0] xo = datetime.datetime.strptime(dt,'%Y-%m-%d').date() xx = xo.toordinal() if tx - xx <= ago : # match postcode to location data try: lidx = pcc.index(int(parts[1])) except: lidx = -1 if lidx > -1 : # have a location, get its distance, append to returned data itspos = loc[lidx] itsdist = getdist(mypos[0],itspos[0],mypos[1],itspos[1]) # compact the date tu = datetime.datetime.strftime(xo,'%y%m%d') cs = [itsdist,tu,parts[1],parts[6]] latest.append(cs) cases.clear() for j in latest : cases.append(j) latest = 0 csv = 0 # if there are any cases, format and output information if len(cases) > 0 : return render(o,c,r) else : o = "" c = "no new cases" return[o,c] thelist = btap() if len(thelist[0]) == 0 : print("\n No cases recorded since " + str(ago) + " days ago.\n This may be due to delays in providing data.\n Data sourced from:\n https://data.nsw.gov.au/data/dataset/covid-19-cases-by-location") else: print(thelist[1] + "\n\n" + thelist[0])
PIC
E72:pic::glsl::demo:
SDF fragment shader

Besides the watermark, this image is sdf math, including text decals, shader is here
It is not optimized; there is no bvh. It will cook your device, view at your own risk.

Relies heavily on examples published by:
  • Inigo Quilez
  • Jamie Wong

Prototyped on cpu in Houdini Vex for safety, before finishing it off in KodeLife.
Render above is from the vex version, as its trivial to output an image sequence from Houdini.

Conclusion to this test: AA kills it. Try again later with WebGPU+Vulkan...

Not made for Bahco. Chosen as its a simple product to model, has text and movable parts, had the measurements on-hand.
2010s
PIC
Disney Shanghai Haloween:pic::hou::tvc:
smoke simulation

uncredited
multisolver:pic::hou::demo:
Jemena Dragon:pic::hou::tvc:
Fire simulation

uncredited
Supernova Slith:pic::hou::cine:
Slith fx lookdev
credits
Supernova_Slith

Who made this at Axis Productions

Modelling:
uncredited

Shaders:
Sergio Caires

Rigging:
David Gonzalez
Guilia Dell'Armi
Jose Cuenca
Marco Baldi
Mario Aquaro

Animation/Previz:
Kevin McDade
Luke Smith
Si Qi Ho
Steven Graham
Thorsten Kesse
Tracey Chung

FX:
Craig Brown
Hudson Martin
Jayden Patterson
Ola Hamlesten

Lighting/Compositing:
Abby Bar
Daniel Schindler
John Barclay
Romain Ferchat

Editing:
Louise McGreggor

Art Direction:
Ben Craig
GreyGoo Battle:pic::hou::cine:
Weapon fx, smoke, rbd sim, fluid sim

uncredited
wiresolver:pic::hou::demo:
firetrail:pic::hou::demo:
CloneWars comets:pic::tvs:
Comet modeling, shading and scripting, cracked glass model
Lookdev & implementation
credits
Clonewars_Comets

Created and Executive Produced by: George Lucas

Supervising Director: Dave Filoni

Produced by: Cary Silver

Score by: Kevin Kiner

Original Star Wars Themes and Score by: John Williams

Line Producer: Athena Yvette Portillo

Supervising Editor or Editor: Jason W.A. Tucker

CG Supervisor - Lighting & FX: Joel Aron

Digital Asset Supervisor: Paul Zinnes

Animation Supervisor: Keith Kellogg

Lead Designer: Kilian Plunkett

Lead Modeler: Chris Bostjanick

Modelers:
John Dodelson
Jon Childress Farmer
Julian Gupner
Corey Weekley

Digital Artists:
Sang Lee
Cassandra Legro

Riggers:
Nicholas Bolden
Franklin Chance
Eddy Piedra

Texture Artists:
Kazunori Aruga
Ken Min
Pouchon Venerin

Lead 3D Story Artist: Daniel Zizmor

Lead Animator: Marchand Jooste

Senior Animators:
Alex Szeto Fai
Gaurav Dubey
JC Wong
Jim Hatibarua
Nirmal David Kolady

Animators:
Ong Geng Biao Alex
Bradley Lorimer
Chiong Ee Jun
Chng Siwei
Chua Han Xiang
Denis Tong Hor Kin
Gan Sze Ching
Jackson Yeoh
Joe Chye
Khet Yee Sang
Mahendra S. Sikarwar
Mohamed Irfan Syarial B Sharif
Norman Chong
Rishi Kaul
Shannan Wong
Vinod Reddy
Viru Sohal
Wong Chin Chiu Chris

Senior Layout Artist: Jebb Ng Ho Ting

Layout Artists:
Irene Koh Ailing
Lai Yeow Kuang
Vincent Sng

Overseas Associate Producer: Aamir Ghani

CG Supervisor: John Kilshaw

Overseas Associate Production Manager: Ellen Xie

Production Coordinators:
Ngan San San
Sarah Tan
Vanessa Seow

Production Assistants:
Christine S. Chan
Irene Cai
Joyce RuiFen Liew
Pearlyn Yeo

Production Accountant: Lynn Tan Si Min

Crowd Technical Director: Prashanth Bhagavan

Effects Lead: Craig Brown

Senior Effects Artist: Jimmy Leung Yin Lai

Effects Artists:
Chan Yuk Leung Mo
Daniel Li Chung Kei
Ricky Yau Kar Yin
Yau Kiu Fung Kaze

Digital Supervisor: Benjamin Radcliffe

Lead Lighting & Compositing Artist: Leo Liao Feng I

Senior Lighting & Compositing Artists:
Eugene Ko Jaehyuk
Jack Yeung Pak Man
Jesse Hildreth
Kathy Chi Sin Yu

Lighting & Compositing Artists:
CY Shih, Eric Tung Wai Chun
Gary K.L. Wong
Lo Wai Lun
Max Mak Tsz Yeung
Oliver Summa
Riuku Chiang Cho Leuk
Tuck Yin Wong
Woon Chi

Matte Painters: Shyam S. Deshpande

Rigging / Pipeline Lead: Seema Gopalakrishnan

Riggers:
Li Feng
C Man
Huang Shicong
Jiet Shern Neo

Lead Asset Artist: Josh Robinson

Senior Asset Artists:
Bobby Yoonsung Jeong
Harsh Borah

Asset Artists:
Low Hiang Yong Ron
Rajbir Singh Dhalla
Ridwan Chandra Choa
Zeng XiangJun

Editor: Nic Anastassiou

Associate Editor: Mathias Hilger

1st Assistant Editor: Kevin Stermer

2nd Assistant Editor: Nate Cormier

Music Editor: Dean Menta

Additional Music by:
Takeshi Furukawa
David Russell
Matthew St. Laurent

Colorist: Sean Wells

Story & Script Development: Steve Tzirlin

Script & Continuity Assistant Production Manager: Gary Scheppke

Script Coordinator: Darci DuBose

Casting Coordinator: Meagan Finnerty

Look Development Production Coordinator: Liz Marshall

Look Development Production Assistant: Megan Engle

Asset Production Supervisor: David Gray

Asset Production Coordinator: Betsy DeHont

Asset Production Assistant: Andy Heitz

3D Story Production Supervisor: Kristine Donovan

3D Story Production Coordinator: Sarah Lister

3D Story Production Assistant: Alex Spotswood

Animation, Lighting, & FX Coordinator: Julie Osborn

Animation & Cloth Production Assistants:
Caitlin Satchell
Brendan Szulik

Studio Post Production & Editorial Manager: Trisha Brunner

Editorial Production Assistant: Alyson Pierce

Editorial Technician: Charles Choo Jr.

Manager of Production Accounting: Darren Cowan

Production Accountant: Michael Peters

Studio Operations: Colum Slevin

Executive Assistant to George Lucas: Jane Bay

Assistant to George Lucas: Anne Merrifield

Additional Production Support:
Lynn Bartsch
Rob Bonstin
Diane Caliva
Brooke Chapman Gregory Tremelling Nick Provenzano / Michael Parkinson
Betsy Delis
Rob Gianino
Kara Henander
Clay Brown Gabrielle Levenson Tu-Anh Nguyen Karessa Bowens Anton Agana / Drew Patterson
Jayesh Dalal Lance Engle Dave Peticolas
Stacy Lundin
Herbert Primig
Trish Rush
Nicole Vigil
Rob Wynn

Scoring Mixer: Mark Evans

Dialogue Recording Services by: The L.A. STUDIOS, Inc.

Dialogue Recordist: Cameron Davis

Assistant Dialogue Recordist: Ryan Coursey

Post Production Sound Services provided by Skywalker Sound, a Lucasfilm Ltd. Company, Marin County, California

Supervising Sound Editor: Matthew Wood

Sound Designer: David Acord

Re-Recording Mixer: David Acord

Head of Animation Technologies: Monique Bradshaw

Quality & Control Lead: Angela Traeger

Pipeline Technical Director: Josh Rowe

Assistant Technical Directors:
Hoa Quang Bui
Gary Huang
David H. Lam
Derek Smith
Cooper Welch
Adrian Tan Seng Kee
CherFong Foo
Ding Xian Cong
Oliver S. Magno
Ong Chiak Shi

Technology Coordinator: Ariel Owens-Barham

Technical Assistants:
Doug Jung
Carl Leland Taylor
Eric Weber
Albin Ng
Beverley Joy G. Ang
Cai Weiyu
Chee Jun Ghai
Christian Gloor
Feon Sua Xin Miao
Francis Woon
Lau E. H. Melvin
Lee Jin Long
Lin YuanJing

Head of Software Development: Tommy Burnette

Project Manager:
Jessica A. Taul
Aishwarya RTK Prasad

Associate Project Manager: Chris Yeo Keng Hong

Software Manager: Mike King

Technical Resource Supervisor:
Christopher Marklund
Betty Shaw

Global Pipeline Engineers:
Aurelien Collard
Mark Hopkins
Zou Yan

Senior IS Developer: Senthil Kumaran

IS Developer: Steven Neo Say Bin

Lead Production Engineer: Boon Hean Low

Production Engineers:
Ed Caspersen
Purnell Davis III
Joe Hura
Akanksha Sahu
Stephan Steinbach
Alvin Ho Tsun Chuen

Junior Production Engineer:
Haw Shijun
Tien Vu Ho

Database Administrator: Yoel Ade Susanto

IT Manager:
Joe Giller
Teoh Shian Phan

IT Support:
Alan Malicse
Clayton Siemens
Azmi Mohamed Maniku
Barry Dexter A. Gonzaga
Benj Cabrera III
Brian V. Balundo
Chriciel DeChavez Rosel
Chris Chua Moo Lian
Dennis M. Allarey
Didik Achmadi
E. Michael Brown
Harvey B. Alegrado
Hendra Kieran
Japril Cudilla
Joanne Sze Kit Ying
Joseph Kan
Lim Soo Ghee Anthony
Nic Brown
Noorwallid Osman
Roy Halim
Sean S.R. Yeo
Tan Hua Pui Sebastian
Teo Say Chong
Ye Lynn Kyaw

Rights and clearances by:
Entertainment Clearances, Inc.
Cassandra Barbour
Laura Sevier
CloneWars Saber vs Stun:pic::tvs:
Saber vs stun-ring, saber vs pipe
Lookdev & implementation
credits
Clonewars_Saber_vs_Stun

Created and Executive Produced by: George Lucas

Supervising Director: Dave Filoni

Produced by: Cary Silver

Score by: Kevin Kiner

Original Star Wars Themes and Score by: John Williams

Line Producer: Athena Yvette Portillo

Supervising Editor or Editor: Jason W.A. Tucker

CG Supervisor - Lighting & FX: Joel Aron

Digital Asset Supervisor: Paul Zinnes

Animation Supervisor: Keith Kellogg

Lead Designer: Kilian Plunkett

Lead Modeler: Chris Bostjanick

Modelers:
John Dodelson
Jon Childress Farmer
Julian Gupner
Corey Weekley

Digital Artists:
Sang Lee
Cassandra Legro

Riggers:
Nicholas Bolden
Franklin Chance
Eddy Piedra

Texture Artists:
Kazunori Aruga
Ken Min
Pouchon Venerin

Lead 3D Story Artist: Daniel Zizmor

Lead Animator: Marchand Jooste

Senior Animators:
Alex Szeto Fai
Gaurav Dubey
JC Wong
Nirmal David Kolady

Animators
Chiong Ee Jun
Chng Siwei
Chua Han Xiang
Denis Tong Hor Kin
Gan Sze Ching
Jackson Yeoh
Joe Chye
Mahendra S. Sikarwar
Mohamed Irfan Syarial B Sharif
Norman Chong
Ong Geng Biao Alex
Rishi Kaul
Shannan Wong
Vinod Reddy
Viru Sohal
Wong Chin Chiu Chris

Senior Layout Artist: Jebb Ng Ho Ting

Layout Artists:
Irene Koh Ailing
Lai Yeow Kuang
Vincent Sng

Overseas Associate Producer: Aamir Ghani

CG Supervisor: John Kilshaw

Overseas Associate Production Manager: Ellen Xie

Production Coordinators:
Ngan San San
Sarah Tan
Vanessa Seow

Production Assistants:
Irene Cai
Joyce RuiFen Liew
Pearlyn Yeo

Production Accountant: Lynn Tan Si Min

Senior Cloth Artist: Kah Jeng Cheong
Cloth Artists:
AJ Siytangco
Chelsea Khoo Hui Ming
Jiang Han
K. Kothandaraman
Shiv sundar

Crowd Technical Director: Prashanth Bhagavan

Effects Lead: Craig Brown

Senior Effects Artist: Jimmy Leung Yin Lai

Effects Artists:
Chan Yuk Leung Mo
Daniel Li Chung Kei
Ricky Yau Kar Yin
Yau Kiu Fung Kaze

Digital Supervisor: Benjamin Radcliffe

Lead Lighting & Compositing Artist: Leo Liao Feng I

Senior Lighting & Compositing Artists:
Eugene Ko Jaehyuk
Jack Yeung Pak Man
Jesse Hildreth
Kathy Chi Sin Yu

Lighting & Compositing Artists:
CY Shih
Eric Tung Wai Chun
Gary K.L. Wong
Lo Wai Lun
Max Mak Tsz Yeung
Oliver Summa
Riuku Chiang Cho Leuk
Tuck Yin Wong
Woon Chi

Matte Painters: Shyam S. Deshpande

Rigging / Pipeline Lead: Seema Gopalakrishnan

Riggers:
C Man
Huang Shicong
Jiet Shern Neo

Lead Asset Artists: Josh Robinson

Senior Asset Artists:
Bobby Yoonsung Jeong
Semin Tho

Asset Artists:
Rajbir Singh Dhalla
Zeng XiangJun

Editor: Nic Anastassiou

Associate Editor: Mathias Hilger

1st Assistant Editor: Kevin Stermer

2nd Assistant Editor: Nate Cormier

Music Editor: Dean Menta

Additional Music by:
Takeshi Furukawa
David Russell
Matthew St. Laurent

Colorist: Sean Wells

Story & Script Development: Steve Tzirlin

Script & Continuity Assistant Production Manager: Gary Scheppke

Script Coordinator: Darci DuBose

Casting Coordinator: Meagan Finnerty

Look Development Production Coordinator: Liz Marshall

Look Development Production Assistant: Megan Engle

Asset Production Supervisor: David Gray

Asset Production Coordinator: Betsy DeHont

Asset Production Assistant: Andy Heitz

3D Story Production Supervisor: Kristine Donovan

3D Story Production Coordinator: Sarah Lister

3D Story Production Assistant: Alex Spotswood

Animation, Lighting, & FX Coordinator: Julie Osborn

Animation & Cloth Production Assistants:
Caitlin Satchell
Brendan Szulik

Studio Post Production & Editorial Manager: Trisha Brunner

Editorial Production Assistant: Alyson Pierce

Editorial Technician: Charles Choo Jr.

Manager of Production Accounting: Darren Cowan

Production Accountant: Michael Peters

Studio Operations: Colum Slevin

Executive Assistant to George Lucas: Jane Bay

Assistant to George Lucas: Anne Merrifield

Additional Production Support:
Lynn Bartsch
Rob Bonstin
Diane Caliva
Brooke Chapman Gregory Tremelling Nick Provenzano / Michael Parkinson
Betsy Delis
Rob Gianino
Kara Henander
Clay Brown Gabrielle Levenson Tu-Anh Nguyen Karessa Bowens Anton Agana / Drew Patterson
Jayesh Dalal Lance Engle Dave Peticolas
Stacy Lundin
Herbert Primig
Trish Rush
Nicole Vigil
Rob Wynn

Scoring Mixer: Mark Evans

Dialogue Recording Services by: The L.A. STUDIOS, Inc.

Dialogue Recordist: Cameron Davis

Assistant Dialogue Recordist: Ryan Coursey

Post Production Sound Services provided by: Skywalker Sound, a Lucasfilm Ltd. Company, Marin County, California

Supervising Sound Editor: Matthew Wood

Sound Designer: David Acord

Re-Recording Mixer: David Acord

Head of Animation Technologies: Monique Bradshaw

Quality & Control Lead: Angela Traeger

Pipeline Technical Director: Josh Rowe

Assistant Technical Directors:
Hoa Quang Bui
Gary Huang
David H. Lam
Derek Smith
Cooper Welch
CherFong Foo
Ding Xian Cong
Oliver S. Magno
Ong Chiak Shi

Technology Coordinator: Ariel Owens-Barham

Technical Assistants:
Doug Jung
Carl Leland Taylor
Eric Weber
Albin Ng
Beverley Joy G. Ang
Cai Weiyu
Chee Jun Ghai
Christian Gloor
Feon Sua Xin Miao
Francis Woon
Lau E. H. Melvin
Lee Jin Long
Lin YuanJing

Head of Software Development: Tommy Burnette

Project Manager:
Jessica A. Taul
Aishwarya RTK Prasad

Associate Project Manager: Chris Yeo Keng Hong

Software Manager: Mike King

Technical Resource Supervisor:
Christopher Marklund
Betty Shaw

Global Pipeline Engineers:
Aurelien Collard
Mark Hopkins
Zou Yan

Senior IS Developer: Senthil Kumaran

IS Developer: Steven Neo Say Bin

Lead Production Engineer: Boon Hean Low

Production Engineers:
Ed Caspersen
Purnell Davis III
Joe Hura
Akanksha Sahu
Stephan Steinbach
Alvin Ho Tsun Chuen

Junior Production Engineer:
Haw Shijun
Tien Vu Ho

Database Administrator: Yoel Ade Susanto

IT Manager:
Joe Giller
Teoh Shian Phan

IT Support:
Alan Malicse
Clayton Siemens
Azmi Mohamed Maniku
Barry Dexter A. Gonzaga
Benj Cabrera III
Brian V. Balundo
Chriciel DeChavez Rosel
Chris Chua Moo Lian
Dennis M. Allarey
Didik Achmadi
E. Michael Brown
Harvey B. Alegrado
Hendra Kieran
Japril Cudilla
Joanne Sze Kit Ying
Joseph Kan
Lim Soo Ghee Anthony
Nic Brown
Noorwallid Osman
Roy Halim
Sean S.R. Yeo
Tan Hua Pui Sebastian
Teo Say Chong
Ye Lynn Kyaw

Rights and clearances by:
Entertainment Clearances, Inc.
Cassandra Barbour
Laura Sevier
Racoon City fire:pic::hou::cine:
Fire and smoke simulation

uncredited
imagemodel:pic::hou::demo:
Diablo3 mist:pic::3ds::hou::cine:
Particles

credits
... are here:
https:www.mobygames.com/game/windows/diablo-iii/credits
Diablo3 re-entry:pic::3ds::cine:
Re-entry effect

credits
... are here:
https:www.mobygames.com/game/windows/diablo-iii/credits
foam:pic::hou::demo:
dumb particles animated in UV space
TXT
vex:hou::vex:
mostly SOPs
corner normals:hou::vex:
int nn = neighbourcount(0,$ptnum); int nb = neighbour(0,$ptnum,0); vector nbp = point(0,"P",nb); vector ab = -normalize(nbp - $P); int cave = 1; float pi = 3.14; vector zn = {0.0,1.0,0.0}; vector upd = normalize($N); if (nn > 1) { int nc = neighbour(0,$ptnum,1); vector ncp = point(0,"P",nc); vector ac = normalize($P - ncp); float ang = 0.0; if (upd.z == 1.0) { ang = ((atan2(ac.x, ac.y) - atan2(ab.x, ab.y) + pi * 2) % (pi * 2)) - pi; } if (upd.z == -1.0) { ang = ((atan2(ac.y, ac.x) - atan2(ab.y, ab.x) + pi * 2) % (pi * 2)) - pi; } if (upd.x == -1.0) { ang = ((atan2(ac.z, ac.y) - atan2(ab.z, ab.y) + pi * 2) % (pi * 2)) - pi; } if (upd.x == 1.0) { ang = ((atan2(ab.z, ab.y) - atan2(ac.z, ac.y) + pi * 2) % (pi * 2)) - pi; } if (abs(upd.y) == 1.0) { ang = ((atan2(ac.x, ac.z) - atan2(ab.x, ab.z) + pi * 2) % (pi * 2)) - pi; } $aa = ang; float rr = acos(dot(ab,ac)); matrix3 mx = 1; rr = degrees(rr); if (ang > 0.0) { rr = 360.0 - rr; } rr = -radians(rr); rr = rr * $blend; rotate(mx,rr,upd); ab = ab * mx; zn = -ab; if (ang < 0.0) { zn = -zn; } if($flf == 1 && $ptnum == 0) { zn = -zn; } if($fll == 1 && $ptnum == ($Npt - 1)) { zn = -zn; } if($fln != 0 && $ptnum == $fln) { zn = -zn; } //if (upd.x == 1.0 && ($ptnum % 3) != 1) { zn = -zn; } //if (upd.z == 1.0 && $ptnum == 0) { zn = -zn; } //if (upd.z == -1.0 && $ptnum != 0) { zn = -zn; } } else { zn = normalize(cross(ab,upd)); if (upd.z == -1.0) { zn = -zn; } if($ptnum == ($Npt-1)) { zn = -zn; } if($flf == 1 && $ptnum == 0) { zn = -zn; } } $zz = zn; $yy = upd;
get centroids:hou::vex:
int pp = addpoint(0,@P); vector nn = primuv(0,"N",@primnum,{0.5,0.5,0.0}); setpointattrib(0,"N",pp,nn); removeprim(0,@primnum,1);
normalized distance:hou::vex:
float maxd = 0.0; float mind = 99999.0; vector cp = set(ch("cpx"),ch("cpy"),ch("cpz")); for (int i = 0; i < @numpt; i++) { float d = distance(point(0,"P",i),cp); if (d > maxd) { maxd = d; } if (d < mind) { mind = d; } } @mind = mind; @maxd = maxd; @perp = clamp(((distance(@P,cp) - mind) / (maxd - mind)),0.0,1.0);
offset curve points:hou::vex:
float vv = (@ptnum*1.0)/(@numpt-1.0); vv = ((vv+ch("ofs"))%1.0); vector uvw = set(vv,0,0); @P = primuv(0,"P",0,uvw);
stacker:hou::vex:
int pmax = prim(1,"textindex",0); int pnum = prim(0,"textindex",@primnum); float mo = (ch("ofs") % pnum); int p = pnum+mo; int m = (pmax+2) * @lm * ch("ll"); int x = pnum+mo; for (int i = 0; i<=ch("oo"); i++){ if (rint(p*m)!=x){ x = x + 1; } } //@wtf = x; @Cd = set(((x%m)==0),0,0);
prim in group:hou::vex:
int cc = i@class; string gn = "segment_" + itoa(cc); int hh = inprimgroup(0,gn,@primnum); if (hh == 1) { //do stuff here... }
carpetbombing:hou::vex:
(originally made for a carpet shader)
clamp down nn even further if there's warping.
float sc = 5.0; float spr = sc*0.5; int ten = 500; vector aa = set(0.0,0.0,0.0); for (int i = 0; i <= ten; i++) { float uu = $uv.x; float vv = $uv.y; float rr = random(i); uu = uu - 0.5; vv = vv - 0.5; uu = uu * sc; vv = vv * sc; matrix mx = 1; float dr = (rr * 360.0); dr = radians(dr); rotate(mx,dr,{0,0.0,1.0}); vector bb = set(uu,vv,0.0); bb = bb * mx; uu = bb.x; vv = bb.y; uu = uu + 0.5; vv = vv + 0.5; uu = uu+(((rand(i)*2)-1)*spr); vv = vv+(((rand(i+1)*2)-1)*spr); bb = set(uu,vv,0); float zz = length(bb - {0.5,0.5,0}); zz = 1 - clamp(zz,0.0,1.0); zz = fit(zz,0.45,1.0,0.0,1.0); float nn = anoise(((bb+rand(i+2))*2.0), 4, 0.8, (1-zz)); nn = nn * zz; float msx = (uu >= 0.0) && (uu <= 1.0); float msy = (vv >= 0.0) && (vv <= 1.0); float ms = msx * msy; zz = zz * ms; nn = fit(nn,0.19,0.2,0.0,1.0); nn = nn * ms; ms = ms * nn; bb = set((aa.x * (1-ms)) + (uu * ms), (aa.y * (1 - ms)) + (vv * ms),0.0); aa = bb; } $atlas = aa;
floorboards:hou::vex:
float sc = 2.0; float vg = 6.0; float ug = 2.0; float uu = $uv.x; float vv = $uv.y; uu = uu - 0.5; vv = vv - 0.5; uu = uu * sc; vv = vv * sc; uu = uu + 0.5; vv = vv + 0.5; uu = uu%1.0; vv = vv%1.0; float iv = int(vv * vg)/(vg-1); uu = uu + iv; float iu = int(uu * ug)/(ug-1); float ru = rand(iu); float rv = rand(iv+ru); ru = rand(iu + rv + 2); uu = uu - ((iu - ru)*((1.0/ug)*(ug-1))); vv = vv - ((iv - rv)*((1.0/vg)*(vg-1))); uu = uu%1.0; vv = vv%1.0; vector bb = set(uu,vv,0); $atlas = bb;
colorbleed:hou::vex:
int @ptnum; int n =neighbourcount(0,@ptnum); float maxval = 0.0; for (int i=0; i <= n; i++) { int nn = neighbour(0,@ptnum,i); float cr = point(0,"Cd",nn); cr = cr * rand(nn); if (cr > @Cd.r) { maxval = cr; maxval = (@Cd.r + ((maxval - @Cd.r) * 0.5)); @Cd = set(maxval, maxval, maxval); } }
wetmap:hou::vex:
int h = pcopen(2,"P",@P,0.1,50); float mcd = point(1,"Cd",@ptnum); while (pciterate(h)) { vector ocd = {0,0,0}; pcimport(h,"Cd",ocd); if (ocd.r > mcd) { mcd = ocd.r; } } pcclose(h); @Cd = set(mcd,mcd,mcd);
shelftools:hou::py:
funnel:hou::py:

Last used with Houdini 18, probably obsolete.

# merge all visible sops from one object into another. # useful for converting assets, FBX and Alembics. # by c.p.brown. 2014~2018 import os nos = [] ds = [] dp = [] tn = "" us = hou.selectedNodes() for i in us : if i.type().category().name() == "Object" : nos.append(i) tn = i.name() c = 0 while len(nos) > c : for i in nos : subs = i.children() for s in subs : if s.type().category().name() == "Sop" : if s.type().isManager() == False : if s.isDisplayFlagSet() == True : if (s in ds) != True : ds.append(s) dp.append(s.path()) else: try: if s.isDisplayFlagSet() == True : nos.append(s) except: print("\tinvisible node found") c = (c + 1) c = 0 if len(ds) > 0 : for i in ds : print("harvested display SOP: " + i.name()) print("\t" + dp[c]) c = (c + 1) try: co = hou.node('/obj/funneled') print('found existing bgeo node : ' + co.name() + ', using it...') except: print("bgeo node not found, creating it...") obj = hou.node('/obj') co = obj.createNode("geo", "funneled") xu = co.children() for u in xu : u.destroy() omcr = 0 cvcr = 0 xfcr = 0 sncr = 0 swncr = 0 svcr = 0 swvcr = 0 cvncr = 0 rfcr = 0 bfcr = 0 swcr = 0 smcr = 0 outncr = 0 try: om = hou.node('/obj/funneled/merge_display_sops') print('object merge exists, clearing it...') om.parm("numobj").set(0) except: print('object merge SOP not found, creating it...') om = co.createNode("object_merge", "merge_display_sops") om.setCurrent(True) om.setSelected(True) om.setDisplayFlag(True) om.parm("xformtype").set(1) omcr = 1 c = 1 om.parm("numobj").set(len(ds)) for i in dp : om.parm("objpath" + str(c)).set(i) c = (c + 1) try: xf = hou.node('/obj/funneled/xf_fix') print('transform fix found: ' + xf.name() + ', using it...') if omcr == 1 : xf.setInput(0,om) except: print('transform SOP not found, creating it...') xf = co.createNode('xform', 'xf_fix') xf.setInput(0,om) xfcr = 1 try: cv = hou.node('/obj/funneled/convert_to_poly') print('converter found: ' + cv.name() + ', using it...') if xfcr == 1 : cv.setInput(0,xf) except: print('convert_to_poly SOP not found, creating it...') cv = co.createNode('convert', 'convert_to_poly') cv.setInput(0,xf) cvcr = 1 try: sn = hou.node('/obj/funneled/set_normals') print('normal found: ' + sn.name() + ', using it...') if cvcr == 1 : sn.setInput(0,cv) except: print('normals SOP not found, creating it...') sn = co.createNode('normal', 'set_normals') sn.setInput(0,cv) sncr = 1 try: swn = hou.node('/obj/funneled/normals_switch') print('normals switcher found: ' + swn.name() + ', using it...') if cvcr == 1: swn.setNextInput(cv) if sncr == 1: swn.setNextInput(sn) except: print('switch normals SOP not found, creating it...') swn = co.createNode('switch', 'normals_switch') swn.setNextInput(cv) swn.setNextInput(sn) swncr = 1 try: sv = hou.node('/obj/funneled/set_v') print('normal found: ' + sv.name() + ', using it...') if swncr == 1 : sv.setInput(0,swn) except: print('trail SOP not found, creating it...') sv = co.createNode('trail', 'set_v') sv.parm('result').set(3) sv.setInput(0,swn) svcr = 1 try: swv = hou.node('/obj/funneled/vel_switch') print('velocity switcher found: ' + swv.name() + ', using it...') if swncr == 1: swv.setNextInput(swn) if svcr == 1: swv.setNextInput(sv) except: print('switch velocity SOP not found, creating it...') swv = co.createNode('switch', 'vel_switch') swv.setNextInput(swn) swv.setNextInput(sv) swvcr = 1 try: cvn = hou.node('/obj/funneled/converted') print('Null found: ' + cvn.name() + ', using it...') if swvcr == 1 : cvn.setInput(0,swv) except: print('null SOP not found, creating it...') cvn = co.createNode('null', 'converted') cvn.setInput(0,swv) cvncr = 1 try: rf = hou.node('/obj/funneled/save_bgeo') print('ROP found: ' + rf.name() + ', using it...') except: print('ROP not found, creating it...') rf = co.createNode('rop_geometry', 'save_bgeo') rf.parm('trange').set(1) rf.parm('sopoutput').set('$HIP/bgeo/`opname("..")`/`opname("..")`.$F.bgeo.sc') rf.setInput(0,cvn) rfcr = 1 try: bf = hou.node('/obj/funneled/saved_bgeo') print('file-sop found: ' + bf.name() + ', using it...') except: print('file SOP not found, creating it...') bf = co.createNode('file', 'saved_bgeo') bf.parm('file').set('$HIP/bgeo/`opname("..")`/`opname("..")`.$F.bgeo.sc') bfcr = 1 try: sw = hou.node('/obj/funneled/geo_switcher') print('geo switcher found: ' + sw.name() + ', using it...') if xfcr == 1: sw.setNextInput(xf) if cvncr == 1: sw.setNextInput(cvn) if bfcr == 1: sw.setNextInput(bf) except: sw = co.createNode('switch', 'geo_switcher') sw.setNextInput(xf) sw.setNextInput(cvn) sw.setNextInput(bf) swcr = 1 try: sm = hou.node('/obj/funneled/set_material') print('set_material found: ' + sm.name() + ', using it...') if swcr == 1 : sm.setInput(0,sw) except: print('material SOP not found, creating it...') sm = co.createNode('material', 'set_material') sm.parm('shop_materialpath1').set('/shop/`opname("..")`') sm.setInput(0,sw) smcr = 1 try: outn = hou.node('/obj/funneled/OUT') print('OUT Null found: ' + outn.name() + ', using it...') if smcr == 1 : outn.setInput(0,sm) except: print('OUT null SOP not found, creating it...') outn = co.createNode('null', 'OUT') outn.setInput(0,sm) outncr = 1 outn.setCurrent(True) outn.setSelected(True) outn.setDisplayFlag(True) outn.setRenderFlag(True) acr = omcr + cvcr + xfcr + sncr + swncr + svcr + swvcr + cvncr + rfcr + bfcr + swcr + smcr + outncr if acr > 0 : co.layoutChildren()
nativecam:hou::py:

Last used with Houdini 18, probably obsolete.

# convert alembic camera to native # by cpbrown abc = hou.selectedNodes() ds = [] dp = [] nos = [] for i in abc: if i.type().category().name() == "Object" : nos.append(i) print("selected objects: " + str(nos)) c = 0 while len(nos) > c : for i in nos : subs = i.children() for s in subs : print("checking sub-node type : " + s.type().name()) if s.type().name() == "cam" : if (s in ds) != True : ds.append(s) dp.append(s.path()) if s.type().category().name() == "Object" : nos.append(s) c = c + 1 if len(ds) > 0 : c = 0 obj = hou.node('/obj') ch = 0 try: ch = hou.node('/obj/cam_convert') print("chopnet found, using it : " + ch.path()) xu = ch.children() for u in xu: u.destroy() except: print("chopnet not found, creating it...") ch = obj.createNode("chopnet", "cam_convert") oo = ch.createNode("ropnet", "save_animation") print("ropnet: " + oo.path()) for i in ds : print("\tharvested camera: " + i.name()) print("\t" + dp[c] + "\n") cc = 0 try: cc = hou.node('obj/native_cam_' + str(c)) print("\tnative camera found, using it : " + cc.path()) except: print("\tnative camera not found, creating it...") cc = obj.createNode("cam", ("native_cam_" + str(c))) print("\t\tnative cam : " + cc.path()) fe = ch.createNode("fetch", ("fetch_cam_parms_" + str(c))) fe.parm('nodepath').set(i.path()) fe.parm('path').set('aperture focal') ge = ch.createNode("object", ("get_cam_xf_" + str(c))) ge.parm('targetpath').set(i.path()) ge.parm('compute').set(8) mg = ch.createNode("merge", ("merge_channels_" + str(c))) mg.setNextInput(ge) mg.setNextInput(fe) sm = ch.createNode("null", "LIVE_" + str(c)) sm.setInput(0,mg) fi = ch.createNode("file", ("load_saved" + str(c))) fi.parm('file').set('$HIP/clip/native_cam_' + str(c) + '.bclip') sw = ch.createNode("switch", ("switch_to_saved" + str(c))) sw.setNextInput(sm) sw.setNextInput(fi) ou = ch.createNode("export", "OUT_" + str(c)) ou.setInput(0,sw) ou.parm('nodepath').set(cc.path()) ou.parm('path').set('tx ty tz rx ry rz sx sy sz aperture focal') ou.setCurrent(True) ou.setSelected(True) ou.setDisplayFlag(True) ou.setExportFlag(True) bc = oo.createNode("channel", ("save_bclip_" + str(c))) bc.parm('choppath').set(sm.path()) bc.parm('chopoutput').set('$HIP/clip/native_cam_' + str(c) + '.bclip') ch.layoutChildren()
objmerge:hou::py:
import glob import os hp = hou.hipFile.path() f = hou.ui.selectFile(multiple_select=True,start_directory=hp,file_type=hou.fileType.Geometry) print("f=" + str(f)) if f != '' : fc = f.split(';') if len(fc) > 0 : c = 0 obj = hou.node("/obj") npn = obj.createNode("geo",fc[0].split('/')[-1:][0].split('.')[0].strip()) xu = npn.children() for u in xu : u.destroy() mrg = npn.createNode("merge") for k in fc: dn = k.split('/')[-1:][0].split('.')[0].strip() print("dn="+dn) nfn = mrg.createInputNode(c,"file") nfn.parm('file').set(k.strip()) c = c + 1 npn.layoutChildren()
deadline:hou::py:
Made in anger.

Originally developed to bypass a whole department of studio 'pipeline' grifters, who later ratfucked it. This sort of problem was fairly common in vfx, and circumventing it could (and still can) get you fired. Regardless, I'd always circumvent as a matter of professional hygiene.

I strongly believe the artist alone should be making decisions about how & when they work. If they get into trouble, they will get help from those they judge to be most effective. These are both important intrinsic skills that improve with use, atrophy with disuse.

Features of this script, that were lacking in the studio pipeline:
  • Send almost any ROP, and any number of selected ROPs to Deadline instantly with 1 click.
  • Not bother the artist with data-entry.
  • Work with vanilla daily-builds of Houdini.
  • Work with Houdini sessions launched however the artist likes.
  • Work with Hipfiles saved anywhere on the network (that the farm can see).
  • Support new ROPs by editing one line.

An pipeline ultimately boils down to a one-liner:
/path/to/program -arg "what the program needs to do"
Which takes surprisingly little effort to assemble. I've even seen artists type these direct commands faster than using a studio pipeline.

With Houdini, you the artist can even bring the chaosandanarchy by doing pipeline with vops; a native pipeline in your hipfile, not a single line of Python :)

Houdini to deadline sample shelftool (very old, obsolete):

# theoretical deadline job-submit shelftool # NOT TESTED: I've not had to do this for ages; have long-since switched to Linux + StuQ # use as reference only # by cpbrown 2016 & 2022 import os import string import getpass import time usr = getpass.getuser() fail = 0 pathtodeadline = "C:\\Program Files\\Thinkbox\\Deadline8\\bin\\deadlinecommand.exe" pathtohoudini = "C:\\hfs\\Houdini " + hou.applicationVersionString() nethfs = "N:\\hfs\\Houdini " + hou.applicationVersionString() ops = hou.selectedNodes() if len(ops) <= 0 : print('select a rop 1st...') fail = 1 for i in ops : otn = i.type().name() if otn != 'ifd' and otn != 'rop_geometry' and otn != 'rop_dop' and otn != 'geometry' and otn != 'wedge' and otn != 'rop_comp' and otn != 'comp' and otn != 'opengl': print('wrong node dude...') fail = 1 if fail == 0 : di = ("C:\\Users\\" + usr + "\\Desktop\\deadline_jobid\\") if not os.path.exists(di): os.mkdir(di) hscene = hou.hipFile.name() hname = hou.hipFile.path().split('.')[0].split('/')[-1] hdir = hou.hipFile.path().split('.')[0].split('/')[-3] deadlinecmd = "" xcmd = ("(xcopy \"" + nethfs + "\" \"" + pathtohoudini + "\\\" /d /e /v /c /r /y /h /i)") for i in ops : otn = i.type().name() opth = i.path() if otn == 'rop_geometry' or otn == 'rop_dop' or otn == 'geometry' or otn == 'wedge' or otn == 'rop_comp' or otn == 'comp' or otn == 'opengl' : opn = i.name() if otn == 'wedge' : drv = i.parm('driver').eval() wd = hou.node(drv) rng = wd.parm('trange').eval() if rng == 0 : fstart = str(int(hou.frame())) fend = fstart fdur = "1" else : fstart = str(int(wd.parm('f1').eval())) fend = str(int(wd.parm('f2').eval())) fdur = str(int(wd.parm('f2').eval() - (wd.parm('f1').eval()-1))) else : rng = i.parm('trange').eval() if rng == 0 : fstart = str(int(hou.frame())) fend = fstart fdur = "1" else : fstart = str(int(i.parm('f1').eval())) fend = str(int(i.parm('f2').eval())) fdur = str(int(i.parm('f2').eval() - (i.parm('f1').eval()-1))) deadlinecmd = "\"" + pathtodeadline + "\" -SubmitCommandLineJob -executable cmd.exe -arguments \"/c <QUOTE>" + nethfs + "\\bin\\hbatch.exe<QUOTE> -c <QUOTE>render " + opth + "<QUOTE> -c quit <QUOTE>" + hou.hipFile.path() + "<QUOTE>\" -frames " + fstart + "-" + fend + " -chunksize " + fdur + " -pool \"hbatch\" -priority 60 -name \"" + hname + "_" + opn + " - hbatch\" -initialstatus \"Active\" -prop MachineLimit=1" if otn == 'ifd' : opn = i.name() print('selected node is a mantra ROP : ' + opn) fstart = str(int(i.parm('f1').eval())) ifdf = i.parm('soho_diskfile').eval() ifdp = ifdf.split('.') if len(ifdp) > 1 : checkp = (ifdp[0] + '.' + fstart + '.' + ifdp[2]) print('checking ifd 1st frame : ' + checkp + '\n') if os.path.isfile(checkp) : rng = i.parm('trange').eval() ifs = i.parm('soho_diskfile').eval() ifa = ifs.split('.') ifs = (ifa[0] + '.<STARTFRAME>.' + ifa[2]) if rng == 0 : fstart = str(int(hou.frame())) fend = fstart else : fstart = str(int(i.parm('f1').eval())) fend = str(int(i.parm('f2').eval())) deadlinecmd = "\"" + pathtodeadline + "\" -SubmitCommandLineJob -executable cmd.exe -arguments \"/c <QUOTE>" + nethfs + "\\bin\\mantra.exe<QUOTE> -V 4a -j 0 -f <QUOTE>" + ifs + "<QUOTE>\" -frames " + fstart + "-" + fend + " -chunksize 1 -priority 60 -name \"" + hname + "_" + opn + "\" -pool \"mantra\" -initialstatus \"Active\" -prop MachineLimit=15" else : print('ifds are missing, write them out 1st...') else : print('ifd path not set...') if deadlinecmd != "" : print(deadlinecmd) j = (di + hname + "_" + opn + "_RENDER_jobID.txt") deadlinecmd = "(" + deadlinecmd + ")>>\"" + j + "\"" if not os.path.isdir(nethfs) : os.system(xcmd) os.system(deadlinecmd)
stuq:hou::py:
Stupidly simple Q
Used to replace Deadline for Indie work in 2017~2018.


Stuq shelftool:
# batch command generator by c.p.brown, 2017~2019 # for encapsulated native projects # hbatch only # linux only import os import string import getpass import time from datetime import datetime from math import sqrt usr = getpass.getuser() pathtohoudini = "/opt/hfs" + hou.applicationVersionString() ops = hou.selectedNodes() # change this to whatever is default on your distro # don't use xterm as it locks-up houdini myfaveterm = "xfce4-terminal" # leading zeroes def prepadstr(instr, pad, padlen) : dif = (padlen - len(instr)) + 1 o = '' for i in range(1,dif) : o = o + pad o = o + instr #print(o) return(o) if len(ops) >= 1 : # get string vars for naming hscene = hou.hipFile.name() hname = hou.hipFile.path().split('.')[0].split('/')[-1].replace('_','-') hdir = hou.hipFile.path().split('.')[0].split('/')[-3].replace('_','-') hj = hou.hipFile.path().split('.')[0].split('/')[-2].replace('_','-') hprj = "/".join(hou.hipFile.path().split('.')[0].split('/')[:-2]) nx = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') print("\nthis project is : " + hprj) # setup the farm pendingdir = hprj + "/farm/pending" runningdir = hprj + "/farm/running" donedir = hprj + "/farm/done" themofile = hprj + "/farm/themo.sh" if not os.path.exists(pendingdir): os.makedirs(pendingdir) if not os.path.exists(runningdir): os.makedirs(runningdir) if not os.path.exists(donedir): os.makedirs(donedir) if not os.path.exists(themofile) : themoscript = """#!/bin/sh while true do iam=`hostname -s` mkdir ./running/$iam/ 2>/dev/null loc=`ls -1 ./pending/$iam/*.sh 2>/dev/null | wc -l` if [ $loc != 0 ]; then file=`ls ./pending/$iam/*.sh | head -n 1` echo $file if [ ! -z $file ]; then if [ -f $file ]; then src=$(readlink --canonicalize $file) fnm=$(basename $src) mv $src ./running/$iam/$fnm /bin/bash ./running/$iam/$fnm mv ./running/$iam/$fnm ./done/$fnm fi fi else file=`ls ./pending/*.sh | head -n 1` echo $file if [ ! -z $file ]; then if [ -f $file ]; then src=$(readlink --canonicalize $file) fnm=$(basename $src) mv $src ./running/$iam/$fnm /bin/bash ./running/$iam/$fnm mv ./running/$iam/$fnm ./done/$fnm fi fi fi file=0 src=0 fnm=0 loc=0 sleep 2 done """ #print("themo.sh doesn't exist, writing it...\n" + themoscript + "\n") # themo.sh doesn't exist... f = open(themofile,"w") print("themo.sh doesn't exist, writing it...\n") for line in themoscript.splitlines(): f.write(line + "\n") f.close() # chmod themo.sh mode = os.stat(themofile).st_mode mode |= (mode & 0o444) >> 2 os.chmod(themofile, mode) # save the scene hup = os.path.dirname(hou.hipFile.name()) + '/' + 'delme_' + nx + '.hip' hou.hipFile.save() cpycmd = "cp " + hou.hipFile.name() + " " + hup print("making a temp file for this job : " + cpycmd) os.system(cpycmd) # copy hfs to a farm-readable directory - yes this means it'll work with daily builds! nethfs = hprj + "/farm/hfs/hfs" + hou.applicationVersionString() + "/" if not os.path.exists(nethfs): os.makedirs(nethfs) if not os.path.exists(nethfs): print("ERROR: couldn't create nethfs dir : " + nethfs) else: xcmd = ("rsync -av " + pathtohoudini + "/ " + nethfs) print("sync hfs : " + xcmd) os.system(xcmd) # mind merged ROPs for i in ops : otn = i.type().name() if otn == 'merge' : for n in i.inputs() : if not n in ops : ops = ops + (n, ) # loop through selected ROPs for i in ops : isasim = 1 fto = '' batchcmd = "" otn = i.type().name() if otn == 'rop_geometry' or otn == 'rop_dop' or otn == 'geometry' or otn == 'wedge' or otn == 'rop_comp' or otn == 'comp' or otn == 'ifd' or otn == 'opengl' or otn == 'prepost': opn = i.name() opth = i.path() print("selected rop " + opn + " is type " + otn) # does it have a host filter? set paths to use it try : fto = i.parm('whitelist').eval() + '/' except : fto = '' if fto == '/' : fto = '' # its a wedge if otn == 'wedge' : drv = i.parm('driver').eval() wd = hou.node(drv) wub = 0 wisasim = 0 # get wedge range, check if custom parms for start & end exist for rendering a subset fstart = 0 fvstart = str(int(i.parm('range1x').eval())) fend = str(int(i.parm('steps1').eval())-1) fvend = str(int(i.parm('range1y').eval())-1) fvdur = str(int(i.parm('range1y').eval() - (i.parm('range1x').eval()-1))) fdur = str(int(i.parm('steps1').eval())) try: fvstart = str(int(i.parm('wubs').eval())) fvend = str(int(i.parm('wube').eval())) fvdur = str(int(i.parm('wube').eval() - (i.parm('wubs').eval()-1))) fend = str(int(i.parm('steps1').eval())-1) fdur = str(int(i.parm('steps1').eval())) wub = 1 except: print("\twedge has no wubs and wube parms, sending the whole thing...") # get wedge-driver range rng = wd.parm('trange').eval() if rng == 0 : sfstart = str(int(hou.frame())) sfend = fstart sfdur = "1" else : sfstart = str(int(wd.parm('f1').eval())) sfend = str(int(wd.parm('f2').eval())) sfdur = str(int(wd.parm('f2').eval() - (wd.parm('f1').eval()-1))) # is the wedge driver a sim? wisasim = 0 try : wisasim = wd.parm('initsim').eval() except: print("wedge driver isn't a sim") if wd.type().name() == 'opengl' : try : wisasim = wd.parm('soho_initsim').eval() except: print("wedge OGL driver isn't a sim") try : if wd.parm('vm_tile_render').eval() == True : wisasim = False except : print("wedge driver has no tile parameter") # get string vars from the wedge driver wpth = wd.path() wpn = wd.name() oldfn = "" wtn = wd.type().name() print("\twedge driver " + wpn + " is type " + wtn) if wtn == 'ifd' : oldfn = wd.parm('vm_picture').eval() if wtn == 'geometry' : oldfn = wd.parm('sopoutput').eval() if wtn == 'rop_geometry' : oldfn = wd.parm('sopoutput').eval() if wtn == 'comp' : oldfn = wd.parm('copoutput').eval() if wtn == 'rop_comp' : oldfn = wd.parm('copoutput').eval() if wtn == 'opengl' : oldfn = wd.parm('picture').eval() ext = oldfn.split(".")[-1] # loop through the wedges oldcc = 0 cc = int(fstart) for n in range(int(fdur)) : wv = (n/((int(fdur)-1) * 1.0)) * (int(fvdur) * 1.0) wv = int(wv) # set wedge range to single_wedge i.parm('wrange').set(1) # set wedge number: i.parm('wedgenum').set(cc) # set per-wedge overrides, since setting wedgenum doesn't work we're just bypassing wedge altogether for non-sims: ch = i.parm('chan1').eval() oldcc = hou.parm(ch).eval() hou.parm(ch).set(wv) print("\t\t\tset channel " + ch + " to value: " + str(wv)) # set wedge driver output manually as there is no wedgeval yet if wtn == 'ifd' : if wd.parm('vm_tile_render').eval() == True : wd.parm('vm_picture').set("$HIP/pic/${OS}/${OS}.pic") else : wd.parm('vm_picture').set("$HIP/pic/${OS}_" + str(cc) + "/${OS}_" + str(cc) + ".$F4.pic") if wtn == 'geometry' : wd.parm('sopoutput').set("$HIP/tmp/${OS}_" + str(cc) + "/${OS}_" + str(cc) + ".$F.bgeo.sc") if wtn == 'rop_geometry' : wd.parm('sopoutput').set("$HIP/tmp/${OS}_" + str(cc) + "/${OS}_" + str(cc) + ".$F.bgeo.sc") if wtn == 'comp' : wd.parm('copoutput').set("$HIP/out/${OS}_" + str(cc) + "/${OS}_" + str(cc) + ".$F4." + ext) if wtn == 'rop_comp' : wd.parm('copoutput').set("$HIP/out/${OS}_" + str(cc) + "/${OS}_" + str(cc) + ".$F4." + ext) if wtn == 'opengl' : wd.parm('picture').set("$HIP/pic/${OS}_" + str(cc) + "/${OS}_" + str(cc) + ".$F4." + ext) if wtn == 'ifd' : print('\t\t\tchecking matra ROP filename: ' + wd.parm('vm_picture').eval()) # save per-wedge hipfile: hou.hipFile.save() whup = os.path.dirname(hou.hipFile.name()) + '/' + 'delme_' + nx + '_wedge_' + str(cc) + '.hip' wcpycmd = "cp " + hou.hipFile.name() + " " + whup os.system(wcpycmd) print("\t\t\tsaved temp hipfile: " + whup) # create batch scripts per frame for non-sims: if not wisasim : print("\t\t\twedge_" + str(cc) + " is not a sim, sending per-frame commands...") scc = int(sfstart) for s in range(int(sfdur)) : pcc = prepadstr(str(scc), "0", len(sfdur)) batchcmd = nethfs + "/bin/hbatch -c \"render -V -f" + str(scc) + " " + str(scc) + " " + wpth + "\" -c \"quit\" " + whup if batchcmd != "" : nn = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') #print(batchcmd) bsh = (hprj + '/farm/pending/' + fto + nn + '_' + pcc + '_' + hdir + '_' + hj + '_' + hname + '_' + wpn + '_' + opn + '_wedge' + str(cc) + '.sh') #print(bsh) if not os.path.exists(os.path.dirname(bsh)): os.makedirs(os.path.dirname(bsh)) f = open(bsh,'w') f.write(batchcmd + '\n') f.close() scc = scc + 1 else: # driver is a sim, send one job per wedge only: print("\t\t\tis a sim, sending one job per wedge") batchcmd = nethfs + "/bin/hbatch -c \"render -V " + wpth + "\" -c \"quit\" " + whup if batchcmd != "" : nn = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') print("\t\t\t" + batchcmd) bsh = (hprj + '/farm/pending/' + fto + nn + '_' + hdir + '_' + hj + '_' + hname + '_' + wpn + '_' + opn + '_wedge' + str(cc) + '.sh') print("\t\t\t" + bsh) if not os.path.exists(os.path.dirname(bsh)): os.makedirs(os.path.dirname(bsh)) f = open(bsh,'w') f.write(batchcmd + '\n') f.close() # reset wedged channel value hou.parm(ch).set(oldcc) # next wedge cc = cc + 1 # reset wedge range mode i.parm('wrange').set(0) # reset the wedge driver file fields to my defaults if wtn == 'ifd' : wd.parm('vm_picture').set("$HIP/pic/${WEDGENUM}_${OS}/${WEDGENUM}_${OS}.$F4.pic") if wtn == 'geometry' : wd.parm('sopoutput').set("$HIP/tmp/${WEDGENUM}_${OS}/${WEDGENUM}_${OS}.$F.bgeo.sc") if wtn == 'rop_geometry' : wd.parm('sopoutput').set("$HIP/tmp/${WEDGENUM}_${OS}/${WEDGENUM}_${OS}.$F.bgeo.sc") if wtn == 'comp' : wd.parm('copoutput').set("$HIP/out/${WEDGENUM}_${OS}/${WEDGENUM}_${OS}.$F4." + ext) if wtn == 'rop_comp' : wd.parm('copoutput').set("$HIP/out/${WEDGENUM}_${OS}/${WEDGENUM}_${OS}.$F4." + ext) if wtn == 'opengl' : wd.parm('picture').set("$HIP/pic/${WEDGENUM}_${OS}/${WEDGENUM}_${OS}.$F4." + ext) # its not a wedge else : if otn != 'prepost' : # its not a utility rop # get ROP ranges rng = i.parm('trange').eval() if rng == 0 : fstart = str(int(hou.frame())) fend = fstart fdur = "1" else : fstart = str(int(i.parm('f1').eval())) fend = str(int(i.parm('f2').eval())) fdur = str(int(i.parm('f2').eval() - (i.parm('f1').eval()-1))) # is it a sim? try : isasim = i.parm('initsim').eval() except: try : isasim = i.parm('soho_initsim').eval() except: isasim = 0 # create batch scripts per frame if not a sim cc = int(fstart) print("is a sim: " + str(isasim)) if not isasim : if otn == 'comp' or otn == 'rop_comp' : # do 10s for COPs, change dv to desired frames-per-job dv = 10 print('is a COP:') cdur = int((int(fdur)) / float(dv)) print('\t'+str(dv)+'-frame job count = ' + str(cdur)) cdreg = (int(fdur))-(cdur*dv) print('\tCOP range = ' + str(int(fdur))) print('\tremainder = ' + str(cdreg)) print('\tsending '+str(dv)+'s...') for n in range(cdur) : cstart= (n*dv)+1 cend= (n*dv)+dv print('\t\tjob ' + str(n) + ' start = ' + str(cstart) + ', end = '+str(cend)) pcc = prepadstr(str(cstart), "0", len(fdur)) batchcmd = nethfs + "/bin/hbatch -c \"render -V -f " + str(cstart) + " " + str(cend) + " " + opth + "\" -c \"quit\" " + hup print('\tcmd = ' + batchcmd) if batchcmd != "" : nn = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') bsh = (hprj + '/farm/pending/' + fto + nn + '_' + pcc + '_' + hdir + '_' + hj + '_' + hname + '_' + opn + '.sh') if not os.path.exists(os.path.dirname(bsh)): os.makedirs(os.path.dirname(bsh)) f = open(bsh,'w') f.write(batchcmd + '\n') f.close() if cdreg > 0 : print('\tsending dregs...') cstart= (cdur*dv) + 1 cend= (int(fdur)) print('\t\tdregs start=' + str(cstart) + ', end = ' + str(cend)) pcc = prepadstr(str(cstart), "0", len(fdur)) batchcmd = nethfs + "/bin/hbatch -c \"render -V -f " + str(cstart) + " " + str(cend) + " " + opth + "\" -c \"quit\" " + hup print('\tcmd = ' + batchcmd) if batchcmd != "" : nn = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') bsh = (hprj + '/farm/pending/' + fto + nn + '_' + pcc + '_' + hdir + '_' + hj + '_' + hname + '_' + opn + '.sh') if not os.path.exists(os.path.dirname(bsh)): os.makedirs(os.path.dirname(bsh)) f = open(bsh,'w') f.write(batchcmd + '\n') f.close() else: for n in range(int(fdur)) : pcc = prepadstr(str(cc), "0", len(fdur)) batchcmd = nethfs + "/bin/hbatch -c \"render -V -f" + str(cc) + " " + str(cc) + " " + opth + "\" -c \"quit\" " + hup print('\tcmd = ' + batchcmd) if batchcmd != "" : nn = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') bsh = (hprj + '/farm/pending/' + fto + nn + '_' + pcc + '_' + hdir + '_' + hj + '_' + hname + '_' + opn + '.sh') if not os.path.exists(os.path.dirname(bsh)): os.makedirs(os.path.dirname(bsh)) f = open(bsh,'w') f.write(batchcmd + '\n') f.close() cc = cc + 1 # create batch script if sim else: batchcmd = nethfs + "/bin/hbatch -c \"render -V " + opth + "\" -c \"quit\" " + hup if batchcmd != "" : nn = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') #print(batchcmd) bsh = (hprj + '/farm/pending/' + fto + nn + '_' + hdir + '_' + hj + '_' + hname + '_' + opn + '.sh') if not os.path.exists(os.path.dirname(bsh)): os.makedirs(os.path.dirname(bsh)) f = open(bsh,'w') f.write(batchcmd + '\n') f.close() # its a utility ROP, just send it, see what happens... else: batchcmd = nethfs + "/bin/hbatch -c \"render -V " + opth + "\" -c \"quit\" " + hup if batchcmd != "" : nn = datetime.strftime(datetime.now(),'%y%m%d_%H%M%S%f') #print(batchcmd) bsh = (hprj + '/farm/pending/' + fto + nn + '_' + hdir + '_' + hj + '_' + hname + '_' + opn + '.sh') if not os.path.exists(os.path.dirname(bsh)): os.makedirs(os.path.dirname(bsh)) f = open(bsh,'w') f.write(batchcmd + '\n') f.close() # execute shell ROPs instantly if otn == 'shell': i.parm('execute').pressButton() time.sleep(1) # run themo if not running already: #import subprocess #pids = subprocess.check_output(["ps","-efl", (hprj + "/farm/*" + themofile)]) pids = -999 # psutil not available... #import psutil #for proc in psutil.process_iter(): # cmds = " ".join(proc.cmds()) # if (hprj + "/farm/*themo.sh") in cmds : # print("themo.sh is already running\n" + str(cmds)) # pids = 0 # break import subprocess ps = subprocess.Popen(['ps', 'efl'], stdout=subprocess.PIPE) pg = subprocess.Popen(['grep', 'themo.sh'], stdin=ps.stdout, stdout=subprocess.PIPE).communicate()[0] ps.stdout.close() if pg.strip() != "" : pids = 1 pg = 0 ps = 0 if pids == 1 : print("themo.sh is already running...") else : print("themo not running, starting it...") os.system(myfaveterm + " -e 'bash -c \"cd " + hprj + "/farm/; ./themo.sh\"'")
Thebeard.py
import os import glob import time from operator import itemgetter from itertools import groupby cdr = os.path.split(os.path.realpath('thebeard.py'))[0] #print('CURRENT DIR = ' + cdr + '\n') rd = cdr + '/running/' pd = cdr + '/pending/' #print('RUNNING DIR = ' + rd + '\n') #print('PENDING DIR = ' + rd + '\n') def padstr(instr, pad, padlen) : #print('FN: padstr(%s,%s,%i)' %(instr, pad, padlen)) dif = (padlen - len(instr)) + 1 o = instr for i in range(1,dif) : o = o + pad #print('\t'+o) return(o) def getjobs(p) : if os.path.isdir(p) : subs = os.listdir(p) #print("subdirs=" + str(subs)) shs = glob.glob(p + '*.sh*') for u in subs : #print("checking: " + p + u) sshs = glob.glob((p + u) + '/*.sh*') shs = shs + sshs #print("shs="+str(shs)) a = [] for s in shs : fp = s.split('_') #print('time = ' + (fp[0]+fp[1]).split('/')[-1:][0]) #print('frame = ' + fp[2]) #print('job = ' + fp[3]) #print('shot = ' + fp[4]) #print('scene = ' + fp[4]) #print('rop = ' + fp[5].split('.')[0]) a.append({'time':(fp[0]+fp[1]).split('/')[-1:][0], 'frame':fp[2], 'job':fp[3], 'shot':fp[4], 'scene':fp[4], 'rop':('_'.join(fp[5:]).split('.')[0])}) return(a) def calcjobs (a) : o = [] sbb = sorted(a, key=itemgetter('time')) maxlen = 0 for okey, ogroup in groupby(sbb, key=lambda x:x['shot']): og = list(ogroup) shotline = (' shot...: ' + okey) shotlen = len(shotline) if shotlen > maxlen : maxlen = shotlen o.append([shotline, str(len(og))]) og = sorted(og, key=itemgetter('scene')) for skey, sgroup in groupby(og, key=lambda x:x['scene']): sg = list(sgroup) sceneline = (' +scene: ' + skey) scenelen = len(sceneline) if scenelen > maxlen : maxlen = scenelen o.append([sceneline, str(len(sg))]) sg = sorted(sg, key=itemgetter('rop')) for rkey, rgroup in groupby(sg, key=lambda x:x['rop']): rg = list(rgroup) ropline = (' +rop.: ' + rkey) roplen = len(ropline) if roplen > maxlen : maxlen = roplen o.append([ropline, str(len(rg))]) return (o, maxlen) def listjobs(a) : o = "" for i in a[0] : pline = padstr(i[0],".",a[1]) o = o + (pline + ' : ' + i[1]) + '\n' return o bb = getjobs(pd) rr = getjobs(rd) jr = calcjobs(rr) jp = calcjobs(bb) lr = listjobs(jr) lb = listjobs(jp) print('running\n' + lr + '\npending\n' + lb)
shotgun:cmd::hou::py:
data
/skullet/GX1/intro/2/anim/whatever/the/hell/you/like | | | | | | | +--- taskname | | +------- shotname | +---------- sequencename +---------------- projectname args
[01] # projectname.............. get from path of file [02] # username................. get from OS [03] # sequencename............. get from path of file [04] # shotname................. get from path of file [05] # taskname................. get from path of file [06] # versionname.............. can be anything, but is required [07] # versioncomment (optional) can be anything [08] # 1stframeofrender......... get from 1st frame of sequence [09] # pathtoshotgunapi......... from shotgun_api3 import shotgun [10] # pathtoffmpeg............. needed to transcode the images
sendittoshotgun:cmd::py:

One-click Shotgun publish.
No data-entry, no bullshit.
Uses daily vanilla Houdini, Shotgun not installed.
Zero api key security.
Linux only.
Last used in 2018, before Autodesk got its hands on SG, so this is probably obsolete.

# direct shotgun publish script # by c.p.brown 2014~2018 # # shotgun should NOT be installed, just grab its 'api' (directory of scripts) # # dependencies - download/make these in a dir called 'sg' # ./sendittoshotgun.py : this script # ./res/python/shotgun_api3 : shotgun api (as of 2018) # ./res/ffmpeg/bin : ffmpeg binaries and resources (can be overridden) # ./res/icons/your_48x48_watermark.png : watermark for lower right of movies (can be overridden) # ./res/fonts/EnvyCode-R.ttf : font for overlay (can be overridden) # # usage # python sendittoshotgun.py "projectname" "sequencename" "shotname" "taskname" "versionname" "username" "fullimagepath" "comment" "fps" # # build this command in a nuke or houdini shelftool, select read-nodes or file-cops, hit the button, keep working... # a sample shelftool should be where you found this script # # minimal dir compliance # /skullet/GX1/intro/2/anim/then/whatever/the/hell/you/like # | | | | # | | | +--- taskname # | | +------- shotname # | +---------- sequencename # +---------------- projectname # # arg sources - there should be ZERO data entry - its the whole point of this excercise # [01] # projectname.............. get from path of file # [02] # username................. get from OS, must match shotgun username! # [03] # sequencename............. get from path of file # [04] # shotname................. get from path of file # [05] # taskname................. get from path of file # [06] # versionname.............. required, can be anything, build automatically, dont' ask user # [07] # versioncomment (optional) can be anything, build automatically, don't ask user by default; make a special shelftool for special people # [08] # 1stframeofrender......... get from 1st frame of sequence # [09] # pathtoshotgunapi......... from shotgun_api3 import shotgun # [10] # pathtoffmpeg............. optional ffmpeg override, eg: /server/tools/ffmpeg/bin/ # [11] # pathtowatermark.......... optional watermark override, must be a .png with alpha # [12] # pathtofont............... optional watermark font override, must be ttf, used for shot info overlay strip # # the srcipt and dependencies are encapsulated within one dir, so it can be copied and used offsite # to save xfer size, have offsite peeps install python and ffmpeg locally, however this may introduce errors, especially with ffmpeg # # NOTE: # this script is probably incompatible with the latest ffmpeg and shotgun_api # use for reference only # also probably won't work on windows. import os import sys import subprocess import shutil import glob import platform from datetime import datetime pn = sys.argv[1] # project name qn = sys.argv[2] # sequence name sn = sys.argv[3] # shot name tn = sys.argv[4] # task name vn = sys.argv[5] # version name un = sys.argv[6] # user name r = sys.argv[7] # 1st frame of render vc = sys.argv[8] # version comment (required, but can be blank) fs = sys.argv[9] # fps # sys.argv[10] is an optional ffmpeg override # sys.argv[11] is an optional watermark override # sys.argv[12] is an optional watermark font override def padz(s,p,l) : dif = (l - len(s)) + 1 o = '' for i in range(1,dif) : o = o + p o = o + s return o dn = datetime.now() dstr = (str(dn.year)[-2:] + padz(str(dn.month),'0',2) + padz(str(dn.day),'0',2)) # this probably won't work in windows... sucks you to be on windows locsg = "./res/" # import shotgun api : pyp = locsg + "python" sys.path.append(pyp) from shotgun_api3 import Shotgun # check it print 'sys.argv[0] = %s' % sys.argv[0] print 'sys.argv[1] = %s' % sys.argv[1] print 'sys.argv[2] = %s' % sys.argv[2] print 'sys.argv[3] = %s' % sys.argv[3] print 'sys.argv[4] = %s' % sys.argv[4] print 'sys.argv[5] = %s' % sys.argv[5] print 'sys.argv[6] = %s' % sys.argv[6] print 'sys.argv[7] = %s' % sys.argv[7] print 'sys.argv[8] = %s' % sys.argv[8] print 'sys.argv[9] = %s' % sys.argv[9] # default to 24fps if its missing if fs == '' : fs = '24' fexe = locsg + "ffmpeg/bin/ffmpeg.exe" fpro = locsg + "ffmpeg/bin/ffprobe.exe" if platform.system() == 'Linux' : fexe = locsg + "ffmpeg/bin/ffmpeg" fpro = locsg + "ffmpeg/bin/ffprobe" if len(sys.argv) > 10 : if os.path.exists(sys.argv[10]) : print "ffmpeg override supplied" fexe = sys.argv[10] fpro = os.path.dirname(sys.argv[10]) + '/ffprobe.exe' if platform.system() == 'Linux' : fpro = os.path.dirname(sys.argv[10]) + '/ffprobe' print '\tsys.argv[10] = %s' % sys.argv[10] wm = locsg + "icons/your_48x48_watermark.png" if len(sys.argv) > 11 : if os.path.exists(sys.argv[11]) : print "watermark override supplied" wm = sys.argv[11] print '\tsys.argv[11] = %s' % sys.argv[11] wmfont = locsg + "fonts/EnvyCode-R.ttf" if len(sys.argv) > 12 : if os.path.exists(sys.argv[12]) : print "watermark font supplied" wmfont = sys.argv[12] print '\tsys.argv[12] = %s' % sys.argv[12] else : print "watermark font not supplied, using default" wmfont = wmfont.replace("\\","/") wmfont = wmfont.replace(":","\\:") # [TODO] opt-out of watermarking watermarkit = 1 # python scripts have zero security, so not even bothering to hide this info # use NDAs, change api keys frequently SERVER_PATH = "https://yourdomain.shotgunstudio.com" SCRIPT_NAME = 'myfistyourface' SCRIPT_KEY = 'this_scripts_api_key' sg = Shotgun(SERVER_PATH, SCRIPT_NAME, SCRIPT_KEY) #sg = 1 if sg != None : #make the mp4: rparts = r.split('.') #print "r = %s" % r # is it a sequence: path/name.####.ext ? if len(rparts) == 3 : rnums = rparts[len(rparts)-2] print "getting 1st frame num..." globie = rparts[0] + '*' + rparts[2] sq = glob.glob(globie) #sq.sort sq.sort(key=lambda x: x.lower()) sqp = sq[0].split('.') sqnums = sqp[len(sqp)-2] startnum = int(sqnums) print "1st frame num = %s" % startnum rs = rparts[0] + ('.'+'%0'+str(len(rnums))+'d.') + rparts[2] rvid = rparts[0] + '.mp4' # get optional lut files from version comments vfcubes = "" if vc != "" : vct = "" vcln = vc.decode("string_escape").splitlines() print "vcln = %s" % vcln for vl in vcln : vlt = vl print "\t\tchecking comment line for lookup path: %s" % vl vp = vl.split('.cube') if len(vp) > 1 : if os.path.exists(vl) : wvl = vl.replace('\\','/') wvl = wvl.replace(':','\\:') vfcubes = vfcubes + 'lut3d=\'' + wvl + '\', ' vlt = '' vct = vct + vlt + '\n' vc = vct #vfcubes = vfcubes.strip(', ') fcmd = "\"" + fexe + "\" -r " + fs + " -thread_queue_size 512 -start_number " + str(startnum) + " -i \"" + rs + "\" -filter_complex \"" + vfcubes + "scale=trunc(in_w/2)*2:trunc(in_h/2)*2\" -pix_fmt yuv420p -vcodec h264 -g 30 -b:v 2000k -vprofile high -bf 0 -y \""+ rvid + "\"" if watermarkit == 1 : imgn = os.path.basename(sys.argv[7]).split('.')[0] probe = fpro + " -i " + r + " -v error -of flat=s=_ -select_streams v:0 -show_entries stream=width,height" print "probing the image : %s" % probe ffs = "18" fpret = subprocess.Popen(probe, stdout=subprocess.PIPE, shell=True) probed = fpret.communicate()[0] print "probe returns : %s" % probed if '=' in probed : wmtxt = dstr + " | " + pn + " | SEQ=" + qn + " | SHOT=" + sn + " | " + imgn + " | " wmtxtlen = len(wmtxt) + 6 tiw = wmtxtlen * 10 print "estimated text width : %s" % tiw iw = int(probed.split('\n')[0].split('=')[1].strip()) #iw = max(iw,int(tiw)) iwf = int((iw / max(1000.0,float(tiw))) * 18) ffs = str(iwf) fbh = int(25.0 * (iwf / 18.0)) + 20 ih = str(int(probed.split('\n')[1].split('=')[1].strip())-fbh) probe = fpro + " -i " + r + " -v error -of flat=s=_ -select_streams v:0 -show_entries stream=sample_aspect_ratio" print "probing it again : %s" % probe fpret = subprocess.Popen(probe, stdout=subprocess.PIPE, shell=True) probed = fpret.communicate()[0] print "probe returns : %s" % probed forcesar = '[0:v]' if '=' in probed : ia = probed.split('\n')[0].split('=')[1].strip() print "found sample aspect : %s" % ia if ia != '' and len(ia.split(':')) > 1 : forcesar = ('setsar=' + ia + ', ') fcmd = "\"" + fexe + "\" -r " + fs + " -thread_queue_size 512 -start_number " + str(startnum) + " -i \"" + rs + "\" -i \"" + wm + "\" -filter_complex \"[0:v]" + forcesar + vfcubes + "scale=trunc(in_w/2)*2:trunc(in_h/2)*2[scale];[scale][1:v]overlay=x=(main_w-overlay_w)-20:y=(main_h-overlay_h)-(" + str(fbh) + "+15),drawbox=y=" + ih + ":color=black@0.3:width=" + str(iw) + ":height=" + str(fbh) + ":t=max,drawtext=fontfile='" + wmfont + "': text=\'" + dstr + " | " + pn + " | SEQ=" + qn + " | SHOT=" + sn + " | " + imgn + " | " + "f %{eif\\:n+1\\:d}\': fontcolor=white@0.5: fontsize=" + ffs + ": x=10: y=(main_h-text_h)-10\" -pix_fmt yuv420p -vcodec h264 -g 30 -b:v 2000k -vprofile high -bf 0 -y \""+ rvid + "\"" #print '\n' print "fcmd = %s" % fcmd fret = subprocess.call(fcmd, shell = True) fret = 1 if fret == 0 : # shotgun... # override username for offsite testing: #un = "cpb" filters = [ ['name','is',pn], ] p = sg.find_one('Project',filters) if p != None : pid = p['id'] #print "local login user is %s" % un filters = [ ['name','is', un], ] u = sg.find_one('HumanUser',filters) if u != None : uid = u['id'] #print "user = %s, sg_humanuser = %s, sg_uid = %s" % (un, u, uid) filters = [ ['project','is', {'type':'Project','id':pid}],['code', 'is', qn] ] q = sg.find_one('Sequence',filters) if q != None : filters = [ ['project','is', {'type':'Project','id':pid}],['code', 'is', sn] ] s = sg.find_one('Shot',filters) if s != None : sid = s['id'] filters = [ ['project','is', {'type':'Project','id':pid}],['entity','is',{'type':'Shot','id':sid}],['content','is',tn] ] t = sg.find_one('Task',filters) if t != None : tid = t['id'] data = { 'project': {'type':'Project','id':pid},'code': (sn+'_'+tn+'_'+vn),'description': vc,'created_by': {'type':'HumanUser', 'id':uid},'sg_path_to_frames': os.path.dirname(r),'sg_status_list': 'rev','entity': {'type':'Shot', 'id':sid},'sg_task': {'type':'Task', 'id':tid},'user': {'type':'HumanUser', 'id':uid} } v = sg.create('Version', data) if v != None : vid = v['id'] rps = r.split('/') # ---------- drive -------- shotgun ------ project ------ scene -------- shot --------- task --------- review reviewpath = rps[0] + '/' + rps[1] + '/' + rps[2] + '/' + rps[3] + '/' + rps[4] + '/' + rps[5] + '/' + "review" + '/' if not os.path.exists(reviewpath) : os.mkdir(reviewpath) reviewvid = reviewpath + os.path.basename(rvid) shutil.copy2(rvid,reviewvid) sg.upload_thumbnail("Version", vid, r) sg.upload("Version", vid,reviewvid,'sg_uploaded_movie') else : print "error: couldn't create new version: %s" % vn else : print "error: can't find task: %s" % tn else : print "error: can't find shot: %s" % sn else : print "error: can't find sequence: %s" % qn else : print "error: user %s isn't in shotgun" % un else : print "error: can't find project: %s" % pn else : print "error: couldn't make the mp4: %s" % rvid else : print "error: unhandled file format, expecting: name.nums.ext" else : print "error: couldn't connect to shotgun"
sample shelftool:hou::py:
# gather args from your scene + selected file-cops, send it to sendittoshotgun,py # by cpbrown 2014~2018 import os import getpass from datetime import datetime import platform import imp import sys import subprocess import shutil import glob def padz(s,p,l) : dif = (l - len(s)) + 1 o = '' for i in range(1,dif) : o = o + p o = o + s return o dn = datetime.now() dstr = (str(dn.year)[-2:] + padz(str(dn.month),'0',2) + padz(str(dn.day),'0',2)) locsg = "/path/to/sg" def sendittoshotgun(project, sequence, shot, task, user, render, version, comment, fps, ffmpeg, water, wfont): if project != '' and shot != '' and sequence != '' and task != '' and user != '' : pyx = locsg + "python/python" sgpy = locsg + "sendittoshotgun.py" qo = "\"" cmd = qo + sgpy + qo + ' ' + qo + project + qo + ' ' + qo + sequence + qo + ' ' + qo + shot + qo + ' ' + qo + task + qo + ' ' + qo + version + qo + ' ' + qo + user + qo + ' ' + qo + render + qo + ' ' + qo + comment + qo + ' ' + qo + fps + qo + ' ' + qo + ffmpeg + qo + ' ' + qo + water + qo + ' ' + qo + wfont + qo return [pyx,cmd] hdir = hou.hipFile.path() hname = hou.hipFile.basename() hp = hdir.split('/') print('hdir='+hdir) # quick test of hipfile location: if len(hp) > 3 : # vars for this scene pn = hp[2] qn = hp[3] sn = hp[4] tn = hp[5] thisdir = os.path.dirname(hdir).split('/')[-1] thisfile = hou.hipFile.name().split('.')[0].split('/')[-1] un=getpass.getuser() print("project== "+pn) print("sequence= "+qn) print("shot===== "+sn) print("task===== "+tn) print("thisdir== "+thisdir) print("thisfile= "+thisfile) print("user===== "+un) r = '' version = '' comment = '' ops = hou.selectedNodes() # loop through each node: for i in ops: otn = i.type().name() if otn == 'file' : version = (dstr + '_' + thisdir + '_' + thisfile + '_' + i.name()) r = i.parm('filename1').eval() comment = (dstr + ': another great update from ' + un + ' via Houdini') print("r======== "+r) print("version== "+version) print("comment== "+comment) print('\n') fps = str(hou.fps()) #send it: sgcmds = sendittoshotgun(pn, qn, sn, tn, un, r, version, comment, fps, "", "", "") print "shotgun command: %s" % sgcmds sgcmd = (sgcmds[0] + ' ' + sgcmds[1]) subprocess.call(sgcmd, shell = True)
REF
smellslikesmoke:cmd::py:
Bushfire alarm, successfully used during the 2019 season.
Required a non-suspending phone OS, used Sailfish... with some D-Bus modifications.
Data sourced from: http:www.rfs.nsw.gov.au/feeds/majorIncidents.xml

Don't repurpose this for future fires: you're roast-pig if you do it wrong.

# this script and notes are for my own personal use only. import urllib.request import xml.etree.ElementTree as et import os import subprocess import re, html from math import sin, cos, sqrt, atan2, radians # the most vunerable point in the evacuation route b = [-33.689990, 150.546212] # sound alarm if any fire poly-points are within this distance to the above # sound a warning if any fire poly-points are within twice this distance # distance based on approx. 20min evac, with fire traveling approx. 30km/h. # however this is risky, fire could be faster, or embers light-up everything way in advance. Check winds and predicted ember-drop zones, move somewhere safer if in doubt. tooclose = 12.0 # fire polys may not be up-to-date so as a fallback check suburb names... # sound alarm if any of these suburbs are mentioned in the xml redburbs = ['faulconbridge', 'springwood', 'linden', 'winmalee', 'valley heights', 'warrimoo', 'woodford', 'bunyan', 'yellow rock', 'mount hay'] # sound a warning if any of these suburbs are mentioned, # sound an alarm if these are described as 'out of control' or 'emergency' yellowburbs = ['tomah', 'bilpin', 'berambing', 'mount banks', 'wentworth falls', 'blaxland', 'glenbrook', 'bowen mountain', 'yarramundi', 'ruined castle'] def getdist(la, lb, oa, ob) : # function posted by gwaramadze & fixed by Michael0x2a on stackexchange, referring to Andrew Hedges, who referred to Bob Chamberlain # https://andrew.hedges.name/experiments/haversine/ # https://stackoverflow.com/questions/19412462/getting-distance-between-two-points-based-on-latitude-longitude # approximate radius of earth in km R = 6371.0 lat1 = lb lon1 = ob lat2 = la lon2 = oa dlon = radians(lon2 - lon1) dlat = radians(lat2 - lat1) a = (sin(dlat / 2) * sin(dlat / 2) + cos(radians(lat1)) * cos(radians(lat2)) * sin(dlon / 2) * sin(dlon / 2)) c = 2 * atan2(sqrt(a), sqrt(1 - a)) distance = R * c return(distance) tag_re = re.compile(r'(<!--.*?-->|<[^>]*>)') cdr = os.path.split(os.path.realpath('redzone.py'))[0] feed = cdr + "/feed.xml" print(feed) ac = [] s = urllib.request.urlretrieve('http://www.rfs.nsw.gov.au/feeds/majorIncidents.xml', feed) t = et.parse(feed).getroot() bb = 'gst-launch-1.0 -q filesrc location=' + cdr + '/jno.wav ! decodebin ! autoaudiosink' aa = 'gst-launch-1.0 -q filesrc location=' + cdr + '/jsu.wav ! decodebin ! autoaudiosink' for d in t.iter('item') : odk = 999.0 tt = "" dd = "" cc = "" isred = False isyellow = False for m in list(d.iter()) : if "title" in m.tag: tt = m.text if "description" in m.tag: dd = no_tags = tag_re.sub('', m.text) dd = html.escape(dd) if "polygon" in m.tag: pp = m.text c = iter(pp.split()) ci = ([x ,next(c, '')] for x in c) o = list(ci) e = [] for k in o : dk = getdist(float(k[0]), b[0], float(k[1]), b[1]) if dk < odk: odk = dk if "category" in m.tag: cc = m.text for r in redburbs : if r in dd.lower() : isred = True for y in yellowburbs : if y in dd.lower() : isyellow = True if isred: print("isred = " + str(isred)) if isyellow or (odk < (tooclose*2) and ("out of control" in dd.lower())): print("{0:.2f}".format(odk) + "Km : " + tt + " fire (" + cc + ")") for q in range(2) : subprocess.call(bb.split()) if isred or odk < tooclose or (isyellow and "emergency" in dd.lower()): print("\n") print("{0:.2f}".format(odk) + "Km : " + tt + " : (" + cc + ")\n\t" + dd + "\n") for q in range(10) : subprocess.call(aa.split())
chaosandanarchy:hou:
screenie
Send stuff to Deadline via VOPs, using daily vanilla Houdini, zero middleware.

Proof of concept... you really don't need a pipeline department to use a task scheduler.
hipfile
org headers:org:
Used for admin work
basic headers:org:
Note: remove the comma at the start of the configuration line.

Ignoreheading prevents the contents from being excluded from the headers, its useful to hide lengthy org-file settings

Org-confirm-babel line just stops emacs from nagging every time you want to eval a src block

Startup entries makes it more readable for large orgfiles.
Options entries forces text/html export to behave, otherwise it makes assumptions...

# -*- mode:org; org-confirm-babel-evaluate: nil; -*- ,* Configuration :ignoreheading: #+STARTUP: indent overview #+STARTUP: align #+STARTUP: inlineimages #+CREATOR: c.p.brown #+OPTIONS: toc:nil num:nil title:nil author:nil #+OPTIONS: ^:nil #+OPTIONS: -:nil #+OPTIONS: \n:t #+OPTIONS: html-postamble:nil
todo headers:org:
set per-orgfile todos

1st line:
# -*- mode:org; org-todo-keyword-faces: (("[0_TODO]" . "orange") ("[1_IP..]" . "yellow") ("[2_FIX.]" . "red") ("[3_WAIT]" . "blue") ("[4_NOPE]" . "black") ("[5_DONE]" . "green")); -*-
then config:
#+TODO: [0_TODO] [1_IP..] [2_FIX.] [3_WAIT] [4_NOPE] [5_DONE]
plaintext latex headers:org:
Surprisingly difficult; org latex export makes a lot of assumptions...

#+LATEX_COMPILER: xelatex #+LATEX_CMD: xelatex #+LATEX_CLASS: koma-article #+LATEX_CLASS_OPTIONS: [a4paper] #+LATEX_HEADER: \usepackage{fontspec} #+LATEX_HEADER: \usepackage{verbatim} #+LATEX_HEADER: \usepackage{ragged2e} #+LATEX_HEADER: \setmainfont[Ligatures=TeX]{Latin Modern Mono} #+LATEX_HEADER: \renewcommand{\seriesdefault}{bx} #+LATEX_HEADER: \hypersetup{colorlinks=true} #+LATEX_HEADER: \usepackage[margin=1cm, ignorefoot, ignoremp, ignorehead]{geometry} #+LATEX_HEADER: \usepackage[utf8]{inputenc} #+LATEX_HEADER: \changefontsizes{14pt} #+LATEX_HEADER: \setlength{\parindent}{0pt} #+LATEX_HEADER: \frenchspacing #+LATEX_HEADER: \newgeometry{right=1cm,left=1cm,top=1cm} #+LATEX_HEADER: \usepackage{fancyhdr} #+LATEX_HEADER: \usepackage{lastpage} #+LATEX_HEADER: \fancyfoot[C]{Page \thepage\ of \pageref{LastPage}} #+LATEX_HEADER: \renewcommand{\headrulewidth}{0pt} #+LATEX_HEADER: \pagestyle{fancy} #+LATEX_HEADER: \usepackage{courier} #+LATEX_HEADER: \setmonofont[Scale=0.8]{Range Mono Medium}
org table:org:
Templates based on admin for vfx work
basic spreadsheet:org:
| item  | cost | qty |    sum |
|-------+------+-----+--------|
| one   | 50.3 |   2 | 100.60 |
| two   | 33.8 |   5 | 169.00 |
| three |  8.6 |   1 |   8.60 |
|-------+------+-----+--------|
| total |      |     | 278.20 |
#+TBLFM: $4=($2*$3);%.2f :: @>$4=vsum(@I$4..@II$4);%.2f
subcategory spreadsheet:org:
#+tblname: with-rownames
| !     | DESCRIPTION   |       SUB |
|-------+---------------+-----------|
| AAA02 | ideas         |  12340.56 |
| BBB15 | words         |  56780.90 |
| CCC35 | actions       |  12340.00 |
| CCC39 | actions       |  56780.00 |
| CCC40 | actions       |   9100.00 |
|-------+---------------+-----------|
|       | 147341.46     |           |
| ^     | income_total  |           |
|-------+---------------+-----------|
| DDD10 | actions       |  -1230.00 |
| DDD20 | actions       |  -4560.00 |
| EEE10 | things        |   -789.62 |
| EEE20 | things        |  -1000.74 |
|-------+---------------+-----------|
|       | -7580.36      |           |
| ^     | expense_total |           |
|-------+---------------+-----------|
| TOTAL |               | 139761.10 |
| %     |               |      95.6 |
#+TBLFM: @1$1=string("!") :: @1$2=string("DESCRIPTION") ::  $income_total=vsum(@I$SUB..@II$SUB);%.2f :: $expense_total=vsum(@III$SUB..@IIII$SUB)  :: @>>$3=vsum(@I$SUB..@IIII$SUB);%.2f ::  @>$3=(1.0 - (abs(@IIII$2) / @II$2)) * 100.0;%.1f
plot spreadsheet:org:
| temp c | temp f |   fn | plot                 | note       |
|--------+--------+------+----------------------+------------|
|    0.0 |   32.0 | 0.00 |                      |            |
|    2.0 |   35.6 | 0.00 |                      |            |
|    4.0 |   39.2 | 0.01 | ▎                    |            |
|    6.0 |   42.8 | 0.01 | ▎                    |            |
|    8.0 |   46.4 | 0.02 | ▍                    |            |
|   10.0 |   50.0 | 0.03 | ▋                    |            |
|   12.0 |   53.6 | 0.05 | █                    | <- heater  |
|   14.0 |   57.2 | 0.08 | █▋                   |            |
|   16.0 |   60.8 | 0.14 | ██▊                  | <- coat    |
|   18.0 |   64.4 | 0.22 | ████▍                |            |
|   20.0 |   68.0 | 0.33 | ██████▋              |            |
|   22.0 |   71.6 | 0.46 | █████████▎           | <- perfect |
|   24.0 |   75.2 | 0.61 | ████████████▎        |            |
|   26.0 |   78.8 | 0.73 | ██████████████▋      |            |
|   28.0 |   82.4 | 0.83 | ████████████████▋    | <- fan     |
|   30.0 |   86.0 | 0.89 | █████████████████▊   |            |
|   32.0 |   89.6 | 0.94 | ██████████████████▊  | <- aircon  |
|   34.0 |   93.2 | 0.96 | ███████████████████▎ |            |
|   36.0 |   96.8 | 0.98 | ███████████████████▋ |            |
|   38.0 |  100.4 | 0.99 | ███████████████████▊ |            |
|   40.0 |  104.0 | 0.99 | ███████████████████▊ |            |
|   42.0 |  107.6 | 1.00 | ████████████████████ |            |
|   44.0 |  111.2 | 1.00 | ████████████████████ |            |
|--------+--------+------+----------------------+------------|
#+TBLFM: $2=(($1*1.8)+32);%.1f :: $3=(1/(1+exp(-($1-(0.0 + ((50.0/10.0)*4.5)))/(50.0/10.0)*0.7)));%.2f :: $4='(orgtbl-uc-draw-cont $3 0.0 1.0 20) 
columnview spreadsheet:org:
useful when you have heaps of properties per item, but only want a few of them in the table
,* alist :PROPERTIES: :COLUMNS: %ITM %COS %QTY :END: ,** anitem :PROPERTIES: :ITM: one :COS: 34.80 :QTY: 3 :URL: storepage.com/one :END: ,** anotheritem :PROPERTIES: :ITM: two :COS: 7.50 :QTY: 12 :URL: storepage.com/two :END: ,** lastitem :PROPERTIES: :ITM: three :COS: 68.00 :QTY: 8 :URL: storepage.com/three :END: ,** line :PROPERTIES: :ITM: - :COS: :QTY: :URL: :END: ,** total :PROPERTIES: :ITM: Total :COS: :QTY: :URL: :END: ,** table #+BEGIN: columnview :id local :skip-empty-rows t | ITM | COS | QTY | |-------+-------+-------| | one | 34.80 | 3 | | two | 7.50 | 12 | | three | 68.00 | 8 | |-------+-------+-------| | Total | | 23.00 | #+TBLFM: @>$3=vsum(@I$3..@II$3);%.2f #+END
haichmonger:cmd:
backup hipfiles and important incidentals
ransacks:
/media/$USER/work/vfx/
/home/$USER/Downloads/
/home/$USER/Documents/
/home/$USER/Desktop/

#!/bin/bash td="$(printf '%s' "${PWD##*/}")" mkdir -p /media/$USER/$td/$USER/H/ mkdir -p /media/$USER/$td/$USER/H/textures/ mkdir -p /media/$USER/$td/$USER/py/ rsync -av --rsync-path="mkdir -p /media/$USER/$td/$USER/H" --prune-empty-dirs --exclude="backup" --exclude="farm" --exclude="delme*.*" --include="*/" --include="*.hip*" --include="*.rat*" --include="*/src/*.*" --exclude="*" /media/$USER/work/vfx/ /media/$USER/$td/$USER/H/$USER_project_backup/ rsync -av --rsync-path="mkdir -p /media/$USER/$td/$USER/H" --prune-empty-dirs --exclude="backup" --exclude="farm" --exclude="delme*.*" --include="*/" --include="*.hip*" --include="*.rat*" --include="*/src/*.*" --exclude="*" /home/$USER/Desktop/ /media/$USER/$td/$USER/H/$USER_project_backup/ rsync -av --rsync-path="mkdir -p /media/$USER/$td/$USER/H" --prune-empty-dirs --exclude="backup" --exclude="farm" --exclude="delme*.*" --include="*/" --include="*.hip*" --include="*.rat*" --include="*/src/*.*" --exclude="*" /home/$USER/Documents/ /media/$USER/$td/$USER/H/$USER_project_backup/ rsync -av --rsync-path="mkdir -p /media/$USER/$td/$USER/H" --prune-empty-dirs --exclude="backup" --exclude="farm" --exclude="delme*.*" --include="*/" --include="*.hip*" --include="*.rat*" --exclude="*" /home/$USER/Downloads/ /media/$USER/$td/$USER/H/download/ find /home/$USER/Desktop/ -name "*.hdr" -type f -size +1024k -exec cp "{}" /media/$USER/$td/$USER/H/textures/ \; find /home/$USER/Downloads/ -name "*.hdr" -type f -size +1024k -exec cp "{}" /media/$USER/$td/$USER/H/textures/ \; find /media/$USER/work/vfx/ -name "*.hdr" -type f -size +1024k -exec cp "{}" /media/$USER/$td/$USER/H/textures/ \; find /home/$USER/Desktop/ -name "*.py" -type f -size +1024k -exec cp "{}" /media/$USER/$td/$USER/py/ \; find /home/$USER/Documents/ -name "*.py" -type f -size +1024k -exec cp "{}" /media/$USER/$td/$USER/py/ \;
indie network:cmd:
network 4 machines via switch
- name server as..: 10.0.0.1 gateway 255.255.255.0 - name client 1 as: 10.0.0.2 gateway 255.255.255.0 - name client 2 as: 10.0.0.3 gateway 255.255.255.0 - name client 3 as: 10.0.0.4 gateway 255.255.255.0 - install ssh server on server: sudo apt install openssh-server - check /etc/hosts, add hosts and ips - check /etc/hosts.allow, add sshd: 10.0.0.2, sshd: 10.0.0.3, sshd: 10.0.0.4 - add a newline at the end of hosts.allow - mkdir ~/Desktop/projectname on all 4 machines - install ssh on clients : sudo apt-get install sshfs - sudo groupadd fuse - sudo adduser $USER fuse - sudo chown $USER:fuse /dev/fuse - sudo chmod +x /dev/fusermount - sshfs $USER@10.0.0.1:/home/$USER/Desktop/projectname ~/Desktop/projectname
fcurves:hou::vex:
hipfile is here
linear:hou::vex:
linear
clamp((@Frame-@in)/@length,0.0,1.0)
Gauss:hou::vex:
gauss
if(@Frame>@in, if(@Frame<(@in+@length),((exp(-((@Frame-(@in+@length))*((@Frame-(@in+@length))/((@in/8.0)*(@length/@in)*@length))))*(@maxv-@minv))+@minv),@maxv), @minv)
Cosine:hou::vex:
cosine
if(@Frame>@in, if(@Frame<(@in+@length), (@minv * (1 - ((1 - cos((@Frame-@in)/(@length/180)))/2))) + (@maxv * ((1 - cos((@Frame-@in)/(@length/180)))/2)), @maxv), @minv)
Cubic:hou::vex:
cubic
Courtesy of Alex McLeod
@cf = clamp(((@Frame-@in)/@length), 0.0, 1.0) (@cf*@cf*@cf*(@cf*(@cf*(-1.69)) + 2.69)) * (@maxv-@minv) + @minv
Exponential:hou::vex:
exponential
1/(1+exp(-(@Frame-(@in + ((@length/10.0)*5.0)))/(@length/10.0)))
houdini cheatsheet:hou::vex:
Misc Expressions for Houdini.

Particle birth = source points:

impulse activation: $FF==1 impulse rate......: npoints(opinputpath(".",0))
Spiral:

tx = cos ($F * 5) * $F / 500 ty = sin ($F * 5) * $F / 500
Map a revolved surface:

($PT % (ch("../revolve1/divs")+1)) / ch("../revolve1/divs")
Interpolate a linear peak from first to last point, per primitive:

0.5 - abs(0.5 - ($PT - ($PR * ($NPT / $NPR))) / ($NPT / $NPR))
Interpolate a linear peak from first to last point:

0.5 - abs(0.5 - ($PT / $NPT))
Interpolate a sine form 1st to last point, per primitive:

sin(($PT - (($NPT / $NPR) * $PR)) / (($NPT / $NPR) / 180))
Interpolate a sine form 1st to last point, per primitive (shorter):

sin(($PT % $PCOUNT) / ($PCOUNT / 180))
Interpolate form first to last point per primitive, given 10 points per primitive and unscrambled point order.

($PT % 11) / 10
Interpolate a sine form 1st to last point:

sin($PT / ($NPT / 180))
Get at the input node variables:

point((opinputpath(".", 0)), $PT, "variable", 0)
Angle between 2 normals:

acos(dot(normalize(vector3($NX, $NY, NZ)), normalize(vector3($NX2, $NY2, $NZ2))))
map an unrolled nurbs sphere, converted to poly @ 2 2 divs:

U = ~($PT % 31) / 30~ V = ~1 - (int(($PT / ($NPT-0)) * 22) / 21)~
map an open torus:

u = ~($PT % (ch("../torus1/cols") + 1))/ch("../torus1/cols")~ v = ~int($PT / (ch("../torus1/cols") + 1))/ch("../torus1/rows")~
spherize geometry:

o = [0,0,0] r = 1.0 q = normalize(P-o) P = o + (q * r)
blend spherized geometry with P using radius:

o = [0,0,0] r = 1.0 q = normalize(P-o) s = o + (q * r) m = clamp((length(P-0)/r), 0.0, 1.0) P = (P * m)+(s * (1-m))
deform geo into cylinder:

o = [0,@P.y,0] r = 1.0 q = normalize(@P-o) c = o + (q * r)
convert normal to euler xyz
connect normal and pos floats to a float_to_matrix vop:

r1.1 = xx r1.2 = xy r1.3 = xz r1.4 = r2.1 = yx r2.2 = yy r2.3 = yz r2.4 = r3.1 = zx r3.2 = zy r3.3 = zz r3.4 = r4.1 = tx r4.2 = ty r4.3 = tz r4.4 = ... then send result to a get-transform vop.

grab world-space position of an object

$TX = vtorigin("", "/obj/geo1")[0] $TY = vtorigin("", "/obj/geo1")[1] $TZ = vtorigin("", "/obj/geo1")[2]
rolling

pos.x = $FF*0.1 rot.z = -ch("tx")/(2.0*$PI*ch("radius")) * 360.0
INK
ghost:ink:
ghost
EOF
summary of a 20 year career in vfx

| role                | started | ended |  cvr | pay   | exit               | learned                            |       |       |
|---------------------+---------+-------+------+-------+--------------------+------------------------------------+-------+-------|
| compositor/mograph  |    1998 |  1999 |  2.5 | dunno | quit               | 3DSMax, Digital Fusion             |       | ███   |
| 3D generalist       |    1999 |  2004 |  4.5 | >90k  | quit/fired         | to much to list here               |       | █████ |
| FX specialist       |    2005 |  2008 |  2.0 | >90k  | quit               | Thinking Particles, ICE            |       | ██    |
| 3D generalist       |    2008 |  2009 | -1.7 | >90k  | quit/fired         | stay away from Hollywood           |    ██ |       |
| 3D generalist       |    2009 |  2009 | -1.8 | >80k  | quit/fired         | Houdini                            |    ██ |       |
| FX specialist       |    2010 |  2011 | -4.3 | >90k  | quit/fired         | rank & departments are retarded    | █████ |       |
| FX vendor           |    2012 |  2012 | -0.3 | >90k  | contract complete  | Houdini simulations                |     █ |       |
| ██ ██████████       |    ████ |  ████ |  2.8 | █████ | ███████ █████████  | █████, ███████, ████████ & ███████ |       | ███   |
| FX vendor           |    2014 |  2015 |  0.5 | >130k | contracts complete | Vex, deadline                      |       | █     |
| FX specialist       |    2015 |  2015 | -4.3 | >90k  | contract complete  | Emacs Org-Mode                     | █████ |       |
| 3D generalist       |    2016 |  2016 | -3.1 | >160k | quit               | 3DSMax still sucks                 |   ███ |       |
| 3D vendor           |    2015 |  2018 |  1.7 | >130k | contracts complete | quoting, negotiating               |       | ██    |
| pipeline specialist |    2016 |  2016 |  2.7 | >130k | contract complete  | Ffmpeg, Shotgun                    |       | ███   |
| pipeline vendor     |    2017 |  2018 | -1.1 | >130k | contract complete  | documentation                      |     █ |       |
|---------------------+---------+-------+------+-------+--------------------+------------------------------------+-------+-------|
#+TBLFM: $8='(orgtbl-uc-draw-cont $4 0.0 -5.0 5) :: $9='(orgtbl-uc-draw-cont $4 0.0 5.0 5)
;;#+TBLFM: $8='(orgtbl-ascii-draw $4 0.0 -5.0 5) :: $9='(orgtbl-ascii-draw $4 0.0 5.0 5)
CVR
cost vs reward

score is +1 for yes, -1 for no, 0 for sometimes,
if the result is below zero, its time to go...

reel-worthy. x 1.0 - results work, won't age badly, can claim credit control how. x 0.9 - respect for professional prerogative control when x 0.8 - doesn't punish efficiency masters..... x 0.8 - worked directly with masters of their craft skills...... x 0.7 - useful new skills good pace... x 0.6 - too slow = boredom, too fast = no sleep pays well... x 0.5 - covers downtime, given 1 dependent & mortgage
In the EOF table above we can see most vfx jobs/shows became a pretty unrewarding churn after 2008, mostly due to aging workforce who formed hierarchies & silos within vfx studios, which then transformed a rewarding profession into punishing servitude... hence all the quit/fired.

Belatedly switching to independent contracting legally resolved the people-problems, but in turn introduced new challenges, mostly inadequate technology relative to its potential and the demands of clients. Also to be fair I should have developed my own tools, and there was technically nothing stopping me (in certain countries), but it was a massive undertaking on-top of an already demanding profession.

The most important lessons from vfx:
  • Great work requires great talent, and great talent is forged in free pursuit of prestige, to succeed or fail by their efforts alone.
  • Prestige is creation of outstanding work, with full professional prerogative, full credit, and payment proportional to its value.
  • Talent includes salespeople (producers). The task at hand is to make and sell great images.
  • A studio can become successful by riding a wave of great talent in its ascent by offering prestige, and access to tools & assistance.
  • Once studio owners start to fuck-around with incumbency, the prestige is stolen by grifters, talent suffers or leaves, studio gets snaked by more focused prestige-driven crews.
  • Fixing a parasitized studio is near impossible, for the same reasons the problem came about: Dependents, mortgages, vices, vanity, and health, all must come before the business, the art, the talent. To fix grift you must endanger what is most important to the grifter, and they will fight like a motherfucker to stop you.
  • An emerging studio has to maintain its focus by avoiding the one pivotal mistake: promoting people. Adopting the delusion of hierarchy, of imaginary perches on an imaginary org-tree... LARPing. Its an expensive mindgame that taxes production, doesn't make or sell artwork, hinders and demoralizes those who do, generally breeds weakness, and certainly isn't of any interest to clients.
  • Bureaucracies may however interest investors, especially those unfamiliar with the actual task of making & selling images. In this way incumbency & grift may become income generating, but not in a sustainable manner; Its like pump'n'dump, or rather: pump'n'drain - money to unproductive incumbents via kpi bonuses. Its not unusual for a studio to secure grants & loans, only to collapse anyway because its bureaucracies strangled the fundamental driver of a creative business: Great talent in free pursuit of prestige.


TL;DR __---- burnouts 0 .'................ `-+-, /| + / | / \. / | -----+/ | talent . +/ | ./ | /......|................. ___--' | | | | | nubs |......|......|................. | test | keep | fire X= Age of experience Y= Success/fail trajectory (deliver on-time, on-spec, unmanaged)
EDIT: as of this update (2025), the above is now totally moot since one director with access to well-trained image synthesis programs can create an entire show: Maximum prestige - despite some harsh criticism about process. I still vividly recall copping a lot of crap from practitioners of 'fine-arts' for using computers back in the day (eg: "why don't you fuckoff to TAFE"), so a similar reaction from practitioners of 'traditional vfx' is expected.
I believe realistic generative animation a good thing as it opens-up moviemaking to anybody with ideas - the craft will become more about one's vision than money & connections (for now). I'm already seeing this with static-image synthesis as artists learned to prompt generation a distinct vision, and now some of those artists are rapidly learning to animate it.
Manual vfx belongs to history/niches now. It was a wild ride, and I'm glad I was lucky enough to experience its most rewarding years from the mid 90's to mid 00's, and also perversely the bad years from the mid 00's onward, as they provided the whole picture of cause & effect, rise & fall...

2000s
PIC
fakeascii:pic::hou::demo:
rectanglepacking:pic::hou::demo:
2012:pic::3ds::film:
Whitehouse & garden: modelling, texturing, lighting, destruction

credits
2012_Whitehouse

Who made this pucture at Scanline LA:
(Camera, original whitehouse model and props were made by 3rd parties)

Matte Painter:
Jay Seo

Modeling and texturing:
Craig Brown (whitehouse and grounds)
David Bryant (background buildings)
Kevin Mains (debris)
Olek Lyzwanski (aircraft carrier)

Animation:
Craig Brown (trees)
Justin Mitchell (vehicles)

Water Simulation:
Andy Byrne
Stephan Trojansky

Lighting:
Craig Brown (whitehouse and grounds)
David Bryant (background buildings)
Andy Byrne (water)
Olek Lyzwanski (aircraft carrier)


Dynamics and particle fx:
Craig Brown (whitehouse and grounds)
Justin Mitchell (ash, small vehicles and debris)

Compositing and 2D effects:
Matt Doll

QC & creative direction:
Stephan Trojansky
Halo:pic::3ds::cine:
Cloak fx

credits
Halo_Cloak

who made this picture at Blur Studio:

Concept Design:
Hugo Martin
Sean McNally
Francisco Ruiz Velasco
Chuck Wojtkiewicz

Matte Painters:
Jaime Jasso
Hugo Martin

Layout:
Franck Balson
David Nibbellin
Nick Whitmire

Mocap Talent:
Gavin Carlton
Steve Gibbons
Heather McKenney
Paul Barthdmsew
Chris Bedrosian
Vanessa Vander Plum
Derron Ross

Mocap Prep and Clean-up:
Ryan Girard

Modeling:
Shaun Absher
Leandro Amaral
Alessandro Baldasseroni
Toni Bratincevic
Corey Butler
Darren Butler
Luis Calero
Zack Cork
Marek Denko
Chris Grim
Jinho Jang
Jaime Jasso
Sze Jones
Tim Jones
Ian Joyner
Kris Kaufman
James Ku
Alex Litchinko
Kevin Margo
Barrett Meeker
Jeremy Strong
Daniel Trbovic

Rigging:
Bryan Hillestad
Enoch Ihde
Michael Stieber
Brian Whitmire

Animation:
Ander Bergstrom
Joseph Chong
Bryan Hillestad
Jeremiah Izzard
Eddie Kim
Michael Loeck
Nick Maw-Naing
Derek Raymond
Davy Sabbe
Emil Simeonov
Peter Starostin
Rini Sugianto
William Vanoost
August Wartenberg
Brian Whitmire
Nick Whitmire
Jeff Wilson

Hair and Cloth Simulation:
Becca Baldwin
Jon Jordan

Lighting and Compositing:
Leandro Amaral
Heikki Anttila
Chris Bedrosian
Toni Bratincevic
Corey Butler
Darren Butler
Jaime Jasso
Tim Jones
Kris Kaufman
Greg Kegel
Patrick Killik
Jan Major
Kevin Margo
Barrett Meeker
Laurent Pierlot
Brian Prince
Fabio Stabel
Daniel Trbovic
Olivier Vernay-Kim
Dave Wilson
Dan Woje

FX:
Craig Brown
Ian Farnsworth
Seung Jae Lee
Andrew Melnychuk-Oseen
Kirby Miller
Brandon Riza
Mark Theriault
Brandon Young

Creative Director:
Tim Miller
Niagara:pic::3ds::cine:
Aurora

credits
Niagara_Aurora

Tim Jones:
Terrain.

C.P.Brown:
Aurora effect, starfield, compositing.
SuperBowl:pic::3ds::tvc:
Morph effect

credits
Superbowl_Morph

who made this picture at Blur Studio:

Mocap:
Ryan Girard

Mocap actor:
Chris Bedrosian

Layout/Animatic:
Andrew Grisdale
David Nibbelin

Rigging:
Mattias Jervill
Adam Swaab

Animation:
Ander Bergstrom
Adam Swaab

Modeling:
Luis Calero
James Ku

Lighting and Compositing:
Adam Swaab
Colin James
Chris Kelley

FX:
Craig Brown
Adam Swaab

Art Direction & QC:
Adam Swaab

Final QC:
Jennifer Miller
Jericho:pic::3ds::cine:
Clouds

credits
Jericho_Cloud

who made this picture at Blur Studio:

Mocap actors:
Gavin Carlton
Christopher Hicks
Heather Miller
Josh Sweeney

Mocap:
Ryan Girard
Jeff Weisend

Layout/Animatic:
Jean-Dominique Fievet

Character Modeling:
Shaun Absher
Luis Calero
Sze Jones
Ian Joyner
Barrett Meeker
Laurent Pierlot

Rigging:
Steve Guevara
Malcolm Thomas-Gustave
Bryan Hillestad

Animation:
Jean-Dominique Fievet
Ryan Girard
Bryan Hillestad
Jacob Patrick
Peter Starostin
Brian Whitmire

Hair and Cloth Simulation:
Rebecca Baldwin
Jon Jordan

Environment and Prop Modeling:
Sebastien Chort
Zack Cork
Daniel Trbovic

Lighting & Compositing:
Luis Calero
Sebastien Chort
Laurent Pierlot
David Stinnett

FX:
Brandon Riza
David Stinnett
c.p.brown

Art direction & QC:
Jerome Denjean

Final QC:
Tim Miller
Empire Earth:pic::3ds::cine:
Rain/character interaction (difficult at the time)

credits
Empire_Earth_Rain

who made this picture at Blur Studio:

Mocap actors:
Chris Bedrosian
Heather McKenney

Mocap:
Ryan Girard
Jeff Weisend

Layout/Animatic:
Andy Grisdale

Modeling:
Shaun Absher
Darren Butler
Sze Jones
Ian Joyner
Alex Litchinko
Barrett Meeker
Iain Morton

Rigging:
Fabio Stabel
Malcolm Thomas-Gustave

Animation:
Andy Grisdale
Davy Sabbe
Nye Warburton

Hair and Cloth Simulation:
Rebecca Baldwin
Jon Jordan

Lighting & Compositing:
Heikki Antilla
Corey Butler
Kris Kaufman
Barrett Meeker
Dave Wilson

FX:
Craig Brown
Seung Jae Lee
Kirby Miller
Brandon Riza
August Wartenburg

Art direction & QC:
Heikki Anttila
Dave Wilson

Final QC:
Tim Miller
imagetofrags:pic::hou::demo:
Sonic Next:pic::3ds::cine:
CG fire

credits
Sonic_Next_Fire

who made this picture at Blur Studio:

Mocap actors:
(? uncredited)

Mocap:
Ryan Girard

Layout/Animatic:
Leo Santos
Jean-Dominique Fievet

Character Modeling:
Sze Jones
Barrett Meeker
Cemre Ozkurt
Dan Rice
Juan Solis

Rigging:
Bryan Hillestad
Steve Guevara
Mattias Jervill
Malcolm Thomas-Gustave

Animation:
Ruel Pascual
Jacob Patrick
Leo Santos
George Schermer
Dave Vallone

Hair and Cloth Simulation:
Jon Jordan
Malcolm Thomas-Gustave

Environment and Prop Modeling:
Tim Jones
Cemre Ozkurt
Dan Rice
Daniel Trbovic

Lighting & Compositing:
Chris Bedrosian
Luis Calero
Sebastien Chort
Joshua Cox
Barrett Meeker
Dan Rice
Daniel Trbovic

FX:
Craig Brown
Jiyoung Hong
Sam Khorshid
Dan Knight

Art direction & QC:
Dan Rice

Final QC:
Tim Miller
Titan Quest:pic::3ds::cine:
Energy effect

credits
Titan_Quest_Telkine

who made this picture at Blur Studio:

Storyboards:
Chuck Wojtkiewicz

Concept Design:
Sean McNally

Matte Painter:
Dylan Cole

Mocap actors:
Steve Gibbons
Christopher Hicks
Vanessa Vander Pluym

Mocap:
Ryan Girard

Layout/Animatic:
Andrew Grisdale
David Nibbellin

Modeling
Shaun Absher
Chris Bedrosian
Luis Calero
Zack Cork
Tim Jones
Ian Joyner
Barrett Meeker
Iain Morton
Laurent Pierlot

Rigging:
Mattias Jervill
Remi McGill

Animation:
Jeff Fowler
Bryan Hillestad
Jacob Patrick
Davy Sabbe
Jason Taylor

Hair and Cloth Simulation:
Jon Jordan
Malcolm Thomas-Gustave

Lighting & Compositing:
Tim Jones
Barrett Meeker
Iain Morton

FX:
Craig Brown
Gus Wartenburg

Art direction & QC:
Tim Jones

Final QC:
Tim Miller
House of Flying Daggers:pic::3ds::film:
Beans

credits
House_of_Flying_Daggers_Beans

Maryanne Lauric: compositing.

Nigel Waddington:
rendering, additional lighting.

C.P.Brown:
animation (pflow setup),
modelling,
texturing and lighting,
scripting (bean randomiser).
Farscape Moya:pic::3ds::tvs:
Moya modelling & texturing, lighting & compositing

credits
Farscape_Moya

Sebastian Olivero: design.

Nigel Waddington: modelling (station).

Josh Simmonds: additional modelling (station).

Jon Thorsen: texturing (station).

Ben West: previs.

Marco Nero: background image.

Serge Kovalenko: moya modelling.

C.P.Brown :
moya modelling and texturing.
additional modelling (station).
animation.
lighting and compositing.
Farscape noseprobe:pic::3ds::tvs:
CG

credits
Farscape_NoseProbe

C.P.Brown :
Design.
Tracking and animation.
Shading.
Compositing.
Farscape starburst:pic::3ds::tvs:
Starburst effect

credits
Farscape_Starburst

Nick Martinelli: set design.

Sebastian Olivero: set design.

Octavio De Lellis: modelling and texturing (hangar).

Ben Malter: Talyn Prop.

Josh Simmonds: smoke shader.

C.P.Brown:
animation and particle fx.
additional texturing (starburst effect).
lighting and compositing.
Farscape frozen:pic::3ds::tvs:
CG

credits
Farscape_Frozen

C.P.Brown :
Camera match and animation.
Modelling and texturing.
Scripting (conditional param linking).
Lighting and compositing.
Farscape city:pic::3ds::tvs:
Lighting, compositing, atmos, shading, modelling, camera.

credits
Farscape_City

Nick Martinelli: Design.

Josh Simmonds: modelling (buildings).

Alex Scollay: modelling (traffic).

Ben Malter:
modelling and texturing
(foreground detail, traffic).
animation (foreground traffic).
fx (foreground fire).

C.P.Brown :
modelling and texturing (buildings, traffic).
camera match.
animation (background traffic).
atmospherics.
lighting and compositing.
Moulin Rouge:pic::3ds::film:
Modelling shading lighting

credits
Moulin_Rouge_TrainStation

Grant Freckleton: Textures.

Nicole Mather: Textures.

kirsty Millar: Compositing.

Chris Cooper:
Final lighting and texturing.
Modelling, animation, smoke.

Ben Malter: Modelling and texturing.

Alex Mcleod: Steam.

C.P.Brown: Modelling, texturing and lighting
Farscape eject:pic::3ds::tvs:
camera animation hair fx lighting

credits
Farscape_Eject

Thomas Kayser:
Matte-painting.

Alex Scolay:
Modeling & texturing.

C.P.Brown:
Camera, animation, fx, hair, lighting.
Farscape Stark:pic::3ds::tvs:
CG

credits
Farscape_Stark

C.P.Brown:
Tracking.
Scripting (3-point tracking helper).
Modelling and texturing.
Lighting and compositing.
Farscape crystals:pic::3ds::tvs:
CG

credits
Farscape_Crystals

C.P.Brown:
Design.
Camera match.
Modelling and texturing.
Lighting and compositing.
TXT
Maxscripts

Use as reference only.
All of these are obsolete, I stopped using 3DSMax in 2011... with a very brief & embarrassing relapse in 2016.

Some scripts are missing, notably:
  • Fura: Fume wedging, sim-queue and previewing tool. Based on an idea by Andrew Melnychuk-Oseen. Held by Blizzard, asked for a release but they refused. Totally not surprised, but still kinda bummed as Fura was one of those useful force-multiplier type tools, also close to being farmable.
  • Nibbler(?): Boolean interface to quickly do art-directable fragmentation. Held by Scanline, didn't bother asking for it since its so specialized.

Pro-tip: Write your scripts at home, publish online, then only use at work if acquisitive inventions articles (invention assignment agreement) are struck out of your contract. Such articles may not be enforceable in your state, but as a precaution you should either:
  • Not develop/use any self-developed technology or novel techniques while an employee.
  • Not work with an employer who insists on such acquisitive agreements, as the experience may be personally and professionally retarding.

None of the following Maxscripts would have been possible without early mentoring/examples by Chris Cooper and Alex Mcleod. Lukas Lepicovsky helped with optimization & organization of heavier scripts later on.

Also: brace yourself for some fat bevles...
pile:mxs::3ds:
screenie
download

Selectionset manager
Scene management functions
Object management functions
One-click scene-state capture/restore capability, similar in principle to Takes in Houdini

Used extensively for specialist FX work over many years
It had become so densely packed with stuff that its difficult to figure-out, especially data capture/restore. I was probably the only one who ever used it!
pile history
Evolution of pile, in chronological order...
pile1
The original, purely a selection-set manager with basic per-set overrides.
Selection sets were chosen over newly introduced Layers for speed & versatility.

pile1

pile2
1st re-write: added object settings, capture/restore of unlimited renderer setting loadouts (3dsmax only had 2).
Data was stored in object user-data, which took more work than globaltracks or an external file, but it was extremely robust: pile set/object settings could be restored from a small number of objects in a heavily decimated scene, or imported 3ds geometry. Data loss was my greatest concern with alternative tools at this time.
My second greatest concern with alternatives was redundant ui: some of these tools literally replicated every renderer parameter in its own ui, which made no sense as we get the params for free with max. In pile I would just capture/restore renderer-settings with 1 click, too easy. It would take me a while to do the same with object/selectionset settings.

pile2

pile5
Pile 3, 4 & 5 gradually introduced params for scene management to cope with 'departments'. Unfortunately in USA viciously gatekept silos were normal, which meant internal revisions (before the client even saw it) plus stressful busybody work to cope with the effects of not being able to pre-negotiate tech-specs directly with fellow artists.
In practical terms this entailed importing geometry/caches and organizing it into render-passes with their required overrides/optimizations, over and over again, sometimes several times per day, right up to the deadline.

By now Pile was getting way too busy, time to re-think the ui design.

pile5

pile6
Re-write, compacted most tools into rc-menus.
Added an expandable side panel for extra utilities like renaming and testrender history.
Finally added 1-click capture/restore for object/selectionset settings.
Capture/restore worked via a hierarchy of needs:
  • scene settings
    • renderer settings (and env)
      • unified state of settings/assignments per selection set
        • per object on/off, visibility, frozen, moblur etc.
These also came with toggles and overrides, plus a maxscript override that could do just about anything.
They were also saved per object, per set... so almost impossible to loose or corrupt the settings without using maxscript to purge them.
Closest thing I've seen to this approach was Takes in Houdini, but those only work for specific params, need ROPs and aren't portable AFIK.

This fully tricked-out capture restore saved me a fuckload of bullshit busybody work so I could focus more on how things looked instead.

pile6

pile7
Removed side panel, condensed the new utilities into as little ui as possible. Everything had to be doable with one or two clicks, no data entry if possible.
Also added diagnostics which helped me speed up many functions, especially renaming for a particular show that was accumulating close to 10k objects that all needed to be organized into sets for carefully staged animation.

Not long afterwards I switched to, or kinda returned to, Houdini - who's sensible ground-up design for VFX negated the need for us to make scene-management tools :D
I was perfectly happy to leave pile & 3ds "fucken" max behind...

pile7

troller:mxs::3ds:
screenie
download

Fast trackview bypass
Useful in heavy scenes where the trackview lags
bag:mxs::3ds:
screenie
download

script repo browser with localize functions
lampr:mxs::3ds:
screenie
download

bake nulls onto the surface of a deforming mesh
mostly used to anchor emitters to pre-cached characters
peel:mxs::3ds:
screenie
download

maxscript versioning maxscript
vertextidy:mxs::3ds:
screenie
download

clean up after booleans
bakeme:mxs::3ds:
screenie
download

bake transforms of selected object
keybaker:mxs::3ds:
screenie
download

bake transforms of selected using another object
facet:mxs::3ds:
screenie
download

make an object look more faceted
not very useful, just trying to replicate something Alex Mcleod did
automata:mxs::3ds:
screenie
download

make brain-coral patterns in vertex colors
not useful, just an experiment to see how it can be done
mapface:mxs::3ds:
screenie
download

align a planar uv gizmo to face selection
zprojector:mxs::3ds:
screenie
download

project verts down onto another mesh
used to make collision geometry
planter:mxs::3ds:
screenie
download

plant selected geometry down onto a mesh
used for set-dressing
bfc:mxs::3ds:
screenie
download

backface culling
used to optimize heavy scenes
INK
buttons:ink:
buttons
effigy:ink:
effigy
ice:ink:
ice
mulch:ink:
mulch
gathering:ink:
gathering
unroll:ink:
unroll
swimmer:ink:
swimmer
component:ink:
component
staple:ink:
staple
actuator:ink:
actuator
shamble:ink:
shamble
ambule:ink:
ambule
bind:ink:
bind
wand:ink:
wand
bone:ink:
bone
twice:ink:
twice
dan:ink:
dan
conduit:ink:
conduit
lattice:ink:
lattice
object:ink:
object
moya:ink:
moya
scrapheap
gage:ink:
gage
elac:ink:
elac
unk1:ink:
unk1
unk2:ink:
unk2
unk3:ink:
unk3
blade:ink:
blade
1990s
INK

Began as deliberate drawings then became automatic. Deliberate drawing felt like boring coloring-in; a chore. Automatic drawing was usually more interesting as I'd be both creator and spectator at the same time.
By the late 90's most drawings were being made half-asleep while waiting for renders.
Hit any :ink: tag to doomscroll the whole lot...

mostly ballpoint, under 10x10cm

animal:ink:
animal
direct:ink:
direct
thinking:ink:
thinking
doctor:ink:
doctor
drill:ink:
drill
bkk:ink:
bkk
plate:ink:
plate
hungry:ink:
hungry
teory:ink:
teory
sampl:ink:
sampl
cyborg:ink:
cyborg
loop:ink:
loop
lush:ink:
lush
batch:ink:
batch
assessment:ink:
assessment
shorts:ink:
shorts
fingers:ink:
fingers
july:ink:
july
stamp:ink:
stamp
feast:ink:
feast
nine:ink:
nine
scrapheap
slime:ink:
slime
Composite
larvae:ink:
larvae
vine:ink:
vine
Had to stop drawing these things.
proto:ink:
proto
Early automatic drawing, in a more growthlike manner.
pill:ink:
pill
lava:ink:
lava
Moving to CG was a return to boring deliberate illustration. However the process of learning how to do stuff mitigated the boredom, for a few years at least.
head:ink:
head
Early automatic drawing, with a deliberate composition.
decap:ink:
decap
80's and early 90's sketchbooks were full of lovely drawings like this, mostly drawn while in captivity at school.
PIC
Student work and demos.
Most Amiga digital work is lost. I had photos (literal screenshots) in storage, somewhere, 10 years ago.
Also lost a bunch of early commercial work, that had to be mastered direct to vhs in a hurry. CD burners weren't really a thing back then.
3dsmax_studies:pic::3ds::demo:
3dsmax_studies
Learning 3DSMax
Textures for top two images courtesy of PhotoAccess Canberra.

Surprisingly easy transition from Prisms.
The wealth of plugins & comfort-features made it the program of choice for about a decade, until Houdini began to surpass it.

Definitely not nostalgic about 3DSMax, too many memories of manually bashing stuff together in a hurry, only to redo it again over and over: every time source changes, shot after shot, show after show... if I was lucky I could get home before dawn.
Decided to risk automating stuff with Maxscript (especially pile), which eventually paid-off, but the long process came with a heap of criticism from peers for "fucking-around".
When Houdini 9 landed it was game-over for Max as far as I was concerned. By then I had been using Houdini at home to replicate difficult effects in a fraction of the time of what it took with Max.
prisms:pic::demo:
prisms
Stills from student animations
They might still on Umatic or DDT somewhere...

Made with Prisms at ACAT Canberra.
We had a choice of Prisms or Houdini V1, the latter having great potential, but just wasn't fast enough.
Still very nostalgic about the Indys, Unix and Prisms, but I'm sure it would wear-off in a few seconds if I had to use it again.
deluxe_paint:pic::demo:
screenie
Scan of a print of a photo of a monitor-screen of pixel-pushing, made with Deluxe Paint on the A500.
Probably from around 1991. The only surviving example I could find.
Obviously inspired by team portraits in SpeedBall2 by The Bitmap Brothers.
ABOUT
Content on this site is a selection of works that are representative of what I was doing at the time, or information I need to keep online for my own reference. Most items are sorted in reverse chronological order.

The :tags: are a toggle to filter & expand items: click once to filter, click it again to unfilter.

  • TXT are scripts or programs I've created and used.
  • REF are notes, derivative works, or instructions for 3rd party technology. REF may also contain some of my abandoned/incomplete works that I need to keep online for my own reference.
  • PIC are artworks I've contributed to in some significant way. Group efforts are accompanied by credits where provided.
  • INK are old drawings. Bit cringe, but I get complaints every time I take them down.


DISCLAIMERS AND DISCLOSURES

  • Information elsewhere on this site is an exercise in fictional creative-writing, created for my own amusement.
  • Programs and code on this site do not come with a license for use, unless otherwise specified in both the code itself and the description of it on this site. If you copy and execute such programs and code regardless, they may cause harm to you, your career, your employer & colleagues, your clients & vendors, your friends & family, innocent bystanders and any related works, possessions and reputations thereof.
  • My programs are not generated by machine-learning (ML) software, as I doubt such practice leads to intrinsic ability. However: Derivative works on this site may or may not have been partly authored by ML, and I often ask ML basic questions on fundamental matters, answers to which may improve or retard my software design techniques.
  • The site is self-contained, doesn't use cookies or analytics: I don't know who visits, don't care.
  • This site was created in Org-Mode, then exported to html using a gifded queue written in Rebol3.