Tag: data liberation

Posted on
by
Calendar interface in Nextcloud

The problem with purism

At heart, I’m a Linux guy.  For many tasks, I use Emacs (a popular editor among some developers due to its extensibility), with Orgmode as my primary means of managing tasks, recording time, jotting down notes and, at times, trying to manage my calendar.

But there were several problems with this. Firstly, the only mobile client to sync Orgmode files with reasonable reliability, was MobileOrg.  Sadly, this project has been discontinued for a while, and to my knowledge it hasn’t yet seen a superior successor.  In addition, Orgmode is a great calendar within Emacs, but it’s not so strong outside. And while MobileOrg was “ok”, it didn’t present information in a convenient, easily-interpreted way.

In short, having a text-only, Linux/Android-only solution, was awkward.

The compromising advantage

Part of the appeal of Orgmode and MobileOrg was being able to keep all data within one’s own infrastructure.  As one of MobileOrg’s features is to “sync files from an SSH server”, and Emacs has TRAMP for accessing network locations, this made it possible to get each end talking with the other, and the synchronisation was generally reliable.

But in some ways, using Emacs, Orgmode and MobileOrg – to achieve data security and ultimate privacy – is arguably a case of the tail wagging the dog.  Was this the only private-data solution? Probably not. Was it the most convenient?  Was Orgmode the right tool for many of life’s repeatable, short-lived events? Definitely not.

image of org-mode
org-mode in action: showing a list of links

Despite trying to use only free, libre & open source software to address this requirement, around 2016 it started becoming clear that simpler solutions existed – albeit involving proprietary software of some kind.  Certain diehards might scoff that, if some software only exists in proprietary form, it’s inherently evil and you must build a free/libre version. But such ideals are rarely achievable when your needs as a new parent and business owner outweigh most others.

As I pondered my motives, it became clear to me that controlling my data was more important to me than controlling the tools.

The next move

For years on Android, I used CalDav and CardDav syncing tools, which were proprietary plugins that presented calendar and contact “providers” to the OS.  These worked great, but finding equivalent staples on Linux was somewhat harder.  The time had arrived when I needed desktop access to calendar, task and contact management, that wasn’t based in an Office365 tenancy.

The right move here was to set up Nextcloud. On my small personal hosting box at DigitalOcean [discount referral link], I set up a virtual server to run Nextcloud.  Nextcloud provides calendar, tasks and contact databases that are conveniently accessible through CardDav & CalDav.

As I had to work on a Mac in order to test websites in Safari (which accounted for at least 9% of traffic, and often more), it was useful to have syncing of this data there too.  And this, unlike some of my earlier grumpiness with all things Mac, was actually a pleasant surprise: macOS actually had great support for CalDav and CardDav.

Conclusion

Account set-up in iOS
Setting up access to other services is a cinch in iOS.

Do I get the solution I need? Yes. Does it sync well? Yes. Am I happier? Yes.

Not only that, but the downside of Orgmode syncing was that it worked best if restricted to two-way communications. If you added a third or fourth client and tried syncing between all of them, it would quickly become a clusterfunk.

Is Apple the enemy?  Well, probably. But better the devil you know, sometimes. Due to the ease of synchronisation with tasks, contacts and calendar in macOS, I slowly warmed up to the idea of replacing my ageing Samsung Galaxy Note 4 with an iPhone. So I did.  And arguably, for this requirement, it was a good choice.

Does this mean I’m no longer a Linux guy? Oh no, not at all. I still have my ThinkPad T420S, which was a side-grade replacement for my chunky T420. I use it every day in my work as a Senior Systems Administrator, for one of the UK’s top universities. I still use Emacs and Orgmode as a daily driver for tasks and coding.

But at home, my wife and I share a calendar and contact list across Android and iOS, thanks to the support of industry standard protocols.

Controlling where the data is has served us pretty well.

Posted on
by
Regain security
Regain email privacy & security

Part #3 of the Data Liberation series

Is there ever time in the day to reconsider your online security? I mean, really consider it?

Take the most common access point for communication in the 21st century – email. Yes, you read that right. It’s still email. Email is the root of online authentication for people worldwide, not only allowing them a “safe place” to recover lost account credentials, but also facilitating properly secured communications with the use of PGP signed and encrypted email. But is your email storage secure?

The woes of web mail

The “problem” with email is that its ubiquity spawned, some years ago, the explosion of “free” web mail services. All the big players provide it. These services are advertising-supported. In other words, the cost of providing such services are met by revenue generated from scanning your email and providing “relevant” adverts within your browser to click on. Each click is tracked and the advertiser billed accordingly.

An issue here, then, is that your email is scanned. All your emails are read by an indexing process which scours every single nugget of information. What information could that include? How could it be used? How about this little list for starters:

  • the date & time
  • the sender’s name and email address
  • their computer’s name
  • their network (i.e. their email provider, their ISP, any intervening mail routers)
  • their probable native language
  • their approximate location when sending the message (obtained from their original IP address)
  • your approximate location when reading the email (based on your IP address)
  • yours and their exact locations if using any location service

That’s not all

If the sender is using the same “free” web-mail service as you:

  • if they use a calendar in that service, what they were doing when they emailed you (giving an insight into the sender’s thought processes…)
  • if they maintain a contact list / address book in that web-mail service, that service may “know” you are a friend or family member of the sender
  • in this case, it will also know their friends – and your friends – and any shared friends too.  It can start to build up a map of contacts – who knows who and possibly why.
  • Knowing “who knows who” means those related contacts’ web-mail services can be interrogated for commonalities, such as shared events (in a calendar), shared interests via a social network, and so on.

Web cam

There are yet more ways your data can be exposed. If they are not using the same “free” web-mail service, but are using another service which they log into using their web mail service’s credentials:

  • that web-mail service provider could poll the other services to see what data you are sending (e.g. what you are posting) to those services
  • it can map any correspondence to or from your contact via its services even when not in relation to your email – e.g. It can expose a contact’s movements, their communications and interests in a given time-frame.
  • they can even be exposed by their use of related services from that provider. For example, new photos into a flickr or instagram account which is public, can be mapped back from their date, time and location to the IP address that was used to query location services.

Finally, a crucial problem with all online services is that there is no guarantee your data is actually deleted when you choose to delete it.  After hitting “delete” through a web site, this could simply flag the email to be removed from your visible account and stored in MegaWebCorp’s vault of “deleted” email, remaining there forever.  Or until needed…

This is the risk of putting data into another provider’s hands – what gets uploaded or stored in your name, stays there in your name, forever.  What goes up, sometimes stays up.

Resolving the privacy crisis

Coming back to email, then, the first priority for someone who wants to maintain some privacy with respect to their life activity needs first to remove the source of indexing from MegaWebCorp’s database – the link between all things you do, your email address.

When the email address is removed from the purview of MegaWebCorp’s systems, your online activity can start to become your business – not the advertiser’s.

Getting your own address is simple.  You can register a domain name with any of numerous providers around the world and sign up for a low-cost hosting plan.  For any person who values their privacy and the sanctity of anonymity, this is a small hurdle to overcome.

For the gain in privacy you can achieve by hosting your own web site, the price attached to a “free” web-mail account may seem rather high.

Bootnote

ArsTechnica has an interesting article published yesterday (30 March 2014) on “metadata as surveillance” .

 

Posted on
by

Part #2 of the Data Liberation series

Mozilla, the organisation behind the ubiquitous Firefox web browser, kindly publishes its source code powering a key service which it provides – Firefox Sync.  Because of this, we are able to run our own password sync servers securely and not necessarily be the target of a large-scale data-mining break-in, such as might be performed by a malicious cracker, or the NSA.  Sorry, of course they are the same thing.

FFirefox logoirefox Sync is a neat service which allows you to, quite literally, sync your settings in Firefox across multiple devices.  These settings can include bookmarks, web browsing history, cookies, form-filling data and passwords.  Anyway, I too was keen to run my own password sync server, so I set about doing just that.

I host quite a bit of stuff using Virtualmin, another superbly produced piece of software which facilitates the set-up of multiple domains on a single box. Setting up Firefox Sync on your own server under virtualmin is actually very straightforward.

The main task at hand is to follow the detailed instructions published by Mozilla.

As per the instructions, I had to run the following, in order to install required software:

# apt-get install python-dev mercurial sqlite3 python-virtualenv libssl-dev

In addition, I also needed to install and enable the WSGI Apache module, which wasn’t present on my system (drawing in dependencies as needed):

# apt-get install libapache2-mod-wsgi

I decided to install the Mozilla sync software in the home directory of my newly created domain, which in Virtualmin is either “/home/domain” or “/home/domain/domains/subdomain”, depending on whether you have created a subdomain for this specific purpose or not.  In the subdomain situation, the folder path would end up being: /home/domain/domains/subdomain/server-full.

Once installed, I inspected the Apache config file. A key change I had to make was to the WSGI configuration within this file. On my Debian box, the Apache config files are located in the standard place: /etc/apache2/sites-available – the same would be true for Ubuntu (on CentOS and other RHEL/Fedora derivatives, you’ll probably find them in /etc/httpd/conf.d/). Once you have created your domain in Virtualmin, your domain’s config file should be within this folder, appropriately named “domain.com.conf”.

In the “domain.com.conf”, there are a few lines to add and one to edit:

Firstly, find the DocumentRoot declaration:

DocumentRoot /home/mydomain/domains/subdomain/public_html

and change it to:
DocumentRoot /home/mydomain/domains/subdomain/server-full

Next, you’ll need to insert the following lines, within the same stanza as DocumentRoot (the best thing is to adjust and paste these lines directly after DocumentRoot:

WSGIProcessGroup sync-http
WSGIDaemonProcess sync-http user=<your-virtualmin-domain's-user> group=<your-virtualmin-domain's-group> processes=2 threads=25
WSGIPassAuthorization On
WSGIScriptAlias / /home/mydomain/domains/subdomain/server-full/sync.wsgi

The above example assumes that you are working within the :80> stanza. If you have enabled SSL on your virtual server, within Virtualmin, then you’ll also have a :443> stanza to add these lines to, with one or two exceptions!

A WSGIDaemonProcess is assigned to each virtual server in Apache. In doing so, it creates a system process which requires a name. According to the WSGI docs, this name must be unique:

“[…] note that the name of the daemon process group must be unique for the whole server. That is, it is not possible to use the same daemon process group name in different virtual hosts. “

When you come to pasting in the additional lines in your :443 stanza, you are dealing with a separate virtual server in Apache.  So, within your Apache config file, be sure to rename your WSGIDaemonProcess process name. E.g.:

WSGIProcessGroup sync-https
WSGIDaemonProcess sync-https user=<your-virtualmin-domain's-user> group=<your-virtualmin-domain's-group> processes=2 threads=25

This configuration should now be valid. You can test this with:

service apache2 reload

This won’t stop the current Apache process, but it will attempt to load the new configuration file. If it fails to load the config, it will tell you without stopping Apache.

Once this works, simply issue:

service apache2 restart

Syncing on mobile

If you intend to use Firefox on Android, or any other mobile Firefox (or clone) that supports the same syncing protocol, there is one caveat.  If you are using an unsigned or self-signed SSL certificate on your sync server, you should visit the site first in your mobile Firefox and add a permanent exception.  Once done, set up firefox sync in the normal way, by typing the characters into your desktop browser’s sync dialog, and the two browsers will shortly be synced up nicely!

Posted on
by

There is a growing trend amongst internet companies – i.e. those organisations who provide services over the internet which store your data – to proclaim your freedom and control over your data. Sometimes, the reality doesn’t quite bear up.

I have decided to write an ad-hoc series of blog posts treating this subject. My main area of focus will be how to use readily-available tools to help you liberate your data and regain control over it.

Keep an eye on my series, at https://dowe.io/tag/data-liberation – and subscribe by email if you want to be kept up-to-date with the latest posts.

Initial plans

The main subjects I am planning to write about at this stage revolve around the current internet/mobile ecosystem and what you can do to live a productive life while maintaining security.

My outline of topics so far:

  • Unlocking your saved passwords from Google Chrome, the internet’s darling web browser
  • Using a free office suite to replace expensive, proprietary vendors’ offerings
  • Getting to grips with your own web account

– why do this? Benefits? – How to set up? – Basic steps for maximum security

  • Using your own internet calendar and contact list, rather than letting your data be snooped on by the easier alternatives…
  • Secure P2P file sharing – no, it’s NOT ILLEGAL!

As well as these practical how-tos, I’m also intending to cover the bigger picture in a few supporting articles:

  • Leaving the “safety” of Windows/MacOS behind. Addressing some misplaced fears.
  • Risks of the “walled garden”
  • Get back in control

– what YOU can do to ensure your rights are not being violated – being pro-active and helping in the community

With writing in mind…

If you would like to suggest ideas or subject areas that you would like covered, please get in touch.

I look forward to your comments!

Posted on
by

This post has a new edition.

Part #1 of the Data Liberation series

Although Google Chrome is a very fast browser, it lacks one key feature which seems designed to lock users in – any account migration facilities to support moving to other browsers.  This post is intended to help you move your saved passwords from Chrome to Firefox.

Firstly, you’ll need to have a read of this page: http://blog.catoblepa.org/2012/08/linux-how-to-export-google-chrome_28.html   – then come back here for more info!

While following the instructions in that post, take note of these steps below before you close your browser. If you have also set up a separate encryption password for your browser, don’t worry – this method still allows access.

  1. Image of Google Chrome settings
    Disconnect Google account in Settings

    In Chrome settings, as a precation, I disconnected my Google account before closing the browser. Therefore, any changes I could make to this temporary session wouldn’t ever be uploaded back to Google.

  2. Once you have the saved CSV file from Chrome, keep hold of it – we need to edit it. In Firefox, install the Password Exporter add-on: https://addons.mozilla.org/en-US/firefox/addon/password-exporter/?src=search
  3. Image of Password Exporter
    Exporting passwords

    Password Exporter allows you to import passwords too, so you can avoid the need to install any third-party workarounds like LastPass (which again require you to upload all your browser data).Firstly, though, using Password Exporter in Firefox (Tools > Add ons … Extensions > Password Exporter > Preferences), we can export a sample CSV file to see how Password Exporter expects its import data. Simply click “Export Passwords” and save the file to your home directory.

    NOTE: This requires that at least one password is saved in Firefox already.

  4. The headings in the exported file are as follows:

hostname username password formSubmitURL httpRealm usernameField passwordField

This is the format that Password Exporter will expect its import data.

The data’s headings that you have just exported from Chrome are a little different:

origin_url action_url username_element username_value password_element password_value submit_element signon_realm ssl_valid preferred date_created blacklisted_by_user scheme password_type possible_usernames times_used

We need to match up the firefox CSV headings with the corresponding Chrome CSV headings. To do this quickly, use a spreadsheet tool I used LibreOffice Calc.

This is what I arrived at:

(FF = Firefox; GC = Google Chrome)

FF: hostname username password formSubmitURL httpRealm usernameField passwordField
GC: origin_url username_value password_value action_url signon_realm username_element password_element

Once the fields are mapped, there’s a couple more important steps to undertake.

Export dialog
Export in the right format!

Firstly, when you come to exporting from your spreadsheet application, make sure you choose to edit the output filter. In the Export Text File dialog, make sure “Quote all text cells” does not have a check (tick) in the box.

For good measure, I also selected ASCII/US in encoding type,  as that is the format used by Password Exporter when exporting.   I think the importer should handle ISO-8859-1 and/or UTF-8, but your mileage may vary.

Now export it.

Remember seeing the additional header in the exported CSV file? It might have looked something like this:

# Generated by Password Exporter; Export format 1.1; Encrypted: false

In order to tell Password Exporter what format to expect its data in, this heading needs to be added back. However… the best way to do this is via a text editor, not in a spreadsheet program.

Open up GEdit, Emacs, Vi… whatever. Add that line to the top, but remove any trailing commas! It should now look like this:

# Generated by Password Exporter; Export format 1.0.4; Encrypted: false
"hostname","username","password","formSubmitURL","httpRealm","usernameField","passwordField"

One more step before you import!

A side-effect of exporting your CSV in LibreOffice is that empty cells are not quoted. In other words, the comma-separated values may appear like this:

"someusername","somepassword","someUrl",,"someusernameField"

Did you see those two commas with nothing between? The Password Exporter won’t like that when trying to import, so do a quick search-and-replace:

Search for ,, and replace with ,””,

Finally, save the file.  Again, ENSURE the file type is US/ASCII.

The importer dialog
Successfully importing passwords!

Now open up the Password Exporter dialog from Firefox and click Import Passwords – you should see progress in the dialog shortly.

CAVEAT #1: BUG WHEN IMPORTING v1.2-EXPORTED DATA

There is an import bug when the version header is declared as 1.1. However, you can get around this by “fudging” the import header to an older version (I used 1.0.4). If you have trouble importing, adjust your header in the file to look like this:

"hostname","username","password","formSubmitURL","httpRealm","usernameField","passwordField"

After importing, you may see that not all passwords were imported. This is because duplicates are not imported. You can view the details in the link.

CAVEAT #2: SOME LOGINS, PASSWORDS, ETC ARE QUOTED

So far I’ve not had time to find a way around this. It’s to do with the import format.

The adventurous can investigate the source code, here: https://github.com/fligtar/password-exporter/blob/master/passwordexporter/chrome/content/pwdex-loginmanager.js

Hopefully you have now successfully liberated your passwords!

Problems?  Comment below!