Blog

The media form field in Joomla 2.5

13th October 2014

Guaranteed to generate a bit of online ire on StackExchange is the Joomla media form field. It doesn't do what it says on the tin; it only allows you to select images from the media manager, not pdfs, audio or video files. That said it does say very clearly on the Joomla docs that it only allows the user to select images, not arbitrary media files. Maybe it should be called images form field or similar. It is however a perfectly reasonable to expect that there will be situations where the user will want to select any media file.

Anyway, you would think that it would be easy to define a custom form field to allow the user to select any media file by extending existing code. Unfortunately, it doesn't seem to be easy at all. In fact on a couple of past projects (mentioning no names) I've committed the cardinal Joomla sin of hacking the core files to stop com_media filtering out files that aren't image files. The list of acceptable file extensions is hard coded into one of the com_media model files. Well, if the Joomla developers are going to do that, I'm going to hack their core.

However, lately I find myself working on a project where the client has set up a Joomla installation on a cloud hosted server where all the installation is done for you, and you can't get at the files on the server other than to upload extensions. So, no core hacks.

So what follows is my solution to getting round this problem with plug-ins only. Firstly caveats; this solution overrides the behaviour of the images view of com_media. It doesn't create a new custom form field leaving the core media form field to behave as described in the Joomla docs. This means that all admin-side uses of the media form field will allow all media files (e.g. in the Images and Links box of the Article editor). If you trust your users not to select videos when they should be selecting images, then this solution suffices. If you want to create a custom field to select all media files and leave the core media form field as is, then that adds another whole layer of complexity, and -- in the words of all the best text-books written by lazy authors -- is left as an exercise for the reader. Second caveat; I make no claims on general applicability. It works for the project I'm currently working on (which is Joomla 2.5). The code is on github if you want to work on it yourself, but I would treat this as a starting point for your project, not a polished solution. Thirdly, I'm first to admit that this solution is a mess, but if anyone can think of something cleaner...

What's the problem?

The core media form field opens a modal dialogue box and iframes the 'images' view of com_media into the dialogue box. You can create your own custom form field which puts different views into the iframe, but the other views defined for com_media don't allow you to select a file. There is no way of parameterising the images view to tell it which file extensions to include in the view; that is hardcoded into a com_media model file. See line 120 of...

administrator/components/com_media/models/list.php

So, possible solutions include creating your own version of com_media (e.g. com_mymedia) where you copy all of the core com_media files into a new component, change its model file and then point your custom form field at a view from that custom component. This is a horrendous amount of work. It is also just about possible to override the images view of com_media using a standard template override. However this does serious damage to the MVC concept because it requires putting a load of code in the layout file that really should be in view.html.php or even in the model file. If, like me, you simply use the default bluestork template for admin, then you'd have to create you own admin template (by copying all of bluestork into your custom template) and then add a template override. Again a solution bordering on insanity.

The solution is...

Lately there has been work showing how you can override core components, so this is the approach I will take. I will override the view.html.php file of the images view of com_media and I also need to override some of the layout files. The changes that actually need making to the view.html.php and layout files are actually quite minor.

All of this takes place in a system plug-in that uses the onAfterRoute event to load in a new version of com_media's images view class that overrides the core one, as follows...

class plgSystemComMediaOverride extends JPlugin {
      public function __construct(&$subject, $config = array()) {
         parent::__construct($subject, $config);
     }

     public function onAfterRoute() {
         $app = JFactory::getApplication();
         if('com_media' == JRequest::getCMD('option') &&
$app->isAdmin()) { require_once(dirname(__FILE__) .
'/code/com_media/views/imageslist/view.html.php'); } } }

This checks to see if we're loading the com_media component and we're in the admin site. If we are, then we require our new view.html.php file.

Note that I have stuck with the new convention of putting these sort of overrides in a directory hierarchy which starts in a directory called code and matches the existing hierarchy of the components and their views. You don't have to, its purely conventional. You could put the view.html.php file anywhere as long as the the require_once command points to it.

Now in the tradition of creating template overrides, you copy the following file...

/administrator/components/com_media/views/imageslist/view.html.php

...to...

WHEREEVER_YOUR_PLUGIN_IS/code/com_media/views/imageslist/view.html.php

...and you can happily edit this copy without damaging the core code.

Getting all the files, not just images

It turns out that the model code used by the imageslist view already gets all the files for you, its just that the imageslist view doesn't use the. The model offers a method getList which returns an array of folders, images and docs, where folders and images are self explanatory, and docs is everything else. The model's getImages method calls getList and then only returns the images part as follows...

	function getImages()
	{
		$list = $this->getList();

		return $list['images'];
	}

(/administrator/components/com_media/models/list.php line 38.)

view.html.php then calls getImages on the model, assigns the images to $images and then calls the layout files which display all the images in $images. So all we have to do is add all the docs to $images and it will then display all the files in the media manager, not just the images. So we replace...

$images = $this->get('images');

...to...

$list = $this->get('list');
$images = array_merge($list['images'], $list['docs']);

...and job done, all files are displayed.

Job not done

Except of course its not quite that simple. The non-image files do not have thumbnail images for them, and the existing layout files attempt to create a thumbnail using the relative path of the file, which the layout files are expecting to be images and now aren't necessarily. So we finish up with output HTML looking like this...

<img src="path/mysong.mp3" alt="..." />

...and depending on your browser you get bad images with question marks, error messages and such like. So we need to assign a thumbnail image to the non-image files. Unfortunately the layout files use the property path_relative to create both the thumbnail and to determine the file that is selected. So if we set path_relative to be the address of some generic thumbnail file, then it would display the thumbnail correctly, but the user would then select the thumbnail image, not the file itself.

Instead of just merging the images and docs together we need to go through them and add a new thumb property which the layout files can use. Now fortunately the model file already assigns icon_32 and icon_16 properties to the docs files based on their extensions. Unfortunately though these properties don't actually point to the images in media/media/images. They nearly do, but not quite. I don't understand this at all, but if you chop off some of address and add some other bits you get the appropriate thumbnail url. Well I did promise this was a bit of a mess, didn't I? Messy code as follows...

    $list = $this->get('list');
    $images = array();
    foreach($list['docs'] as $d) {
        $d->thumb = JURI::root() .
                'media/media/images' .
                substr($d->icon_32, strpos($d->icon_32, '/'));
        $images[] = $d;
    }
    foreach($list['images'] as $i) {
        $i->thumb = COM_MEDIA_BASEURL . '/' . $i->path_relative;
        $images[] = $i;
    }

Finally we need to override the layout files so that they display the thumb property as the thumbnail instead of path_relative. Copy the file at...

/administrator/components/com_media/views/imageslist/tmpl/default_image.php

...to...

WHEREEVER_YOUR_PLUGIN_IS/code/com_media/views/imageslist/tmpl/default_image.php

Then change the line...

<?php echo JHtml::_('image', $this->baseURL.'/'.$this->_tmp_img->path_relative, JText::sprintf('COM_MEDIA_IMAGE_TITLE...

...to...

<?php echo JHtml::_('image', $this->_tmp_img->thumb, JText::sprintf('COM_MEDIA_IMAGE_TITLE...

Now we need to tell the view where to find this override. I'm not sure whether there's a more elegant way of doing this, but the _path['template'] property of the view contains an array of paths to possible layout files, usually containing the template override and the layout files in the component. Presumably the first existing file in the array gets used. So we unshift our override into this array before calling the parent's display method.

    array_unshift($this->_path['template'], dirname(__FILE__) . '/tmpl');
    parent::display($tpl);

So that's all! Working that out is about five hours I won't be getting back again. I hope it helps someone.

http://www.richardbutterworth.co.uk/blog/19-the-joomla-media-form-field

Richard Tomes

1st July 2014

This is a tribute -- of sorts -- to a dear friend and possibility one of the greatest influences on my life: Richard Tomes.

Richard died at the stupidly early age of 46. We hadn't spoken for over eight years; I seemed to have fallen off his invitation list after my daughter was born. Exactly why I didn't really know, and I don't suppose I ever will know now. I suppose I could declare a huge regret in not keeping in touch with him in these last eight years, but that just feels corny somehow. The obvious and trite thing to do, and Richard was never the one for doing the trite and obvious, so yes; regrets, I've had a few, but then again...

Given that barely anyone in my current social/work circle knew Richard, I have had quite a tussle over the past few weeks trying to explain who he was, what he meant to me, and quite why his death had knocked me sideways quite so much. The best one-liner I can come up with is that he was the Sebastian Flyte to my Charles Ryder. Though that doesn't really do our relationship much justice. It doesn't do me much justice either, but a literary reference always makes you look clever.

The Best Days of our Lives?

We were at school together but our paths rarely crossed. He had a reputation for being difficult, troubled and a bit dangerous. I had a reputation for being a weedy and deeply suspect geek, possibly gay. (This was in the days before being a geek -- or gay -- made you fashionable and cool.) He was kicked out of school when he was fifteen or sixteen, for some Dreadful Misdemeanour. Exactly what the Misdemeanour was is pretty much lost in the mists now, he broke something, or shouted at someone, or looked as if he was going to. That was enough in our school. He was shipped across the park to live out his remaining schooldays at the euphemistically titled Special School, and that was the end of his academic career. Our school was a bastard to him, and he carried that with him for as long as I knew him. At one point I was trying to get him to go to an art school, where he would have undoubtedly been brilliant, and which he would have loved, but he couldn't get past it being a school, so that was that. He never went. So well done, our school.

We then finished up in a mutual group of friends in our mid to late teens, and what a revelation to me he was. At least what a revelation his garden was. He'd built a small house at the bottom of it, and a small army of mid pubescent boys and girls would congregate in on a nightly basis, and smoke Bensons and Hedges until our eyes bubbled. He then remodelled it as a Viking long house, with a central fireplace and, slightly inauthentically for a Viking long house, a roulette table. We cared little for historical accuracy in those days, and we continued to blithely bubble away at our Bensons and Hedges. My main memory of that time was his long suffering mum, who would make the nightly journey, ignoring the wind, the rain, the mud, and the historical inauthenticity, into the depths of the garden to politely kick us all out at 11pm on the dot. Poor Janet. We must have absolutely stunk in there.

As is usual for mobs of confused teenagers, everlasting friendships turned into implacable enmities, and I found that our social groupings had shifted and Richard was no longer approved of by my social circle, although I certainly bore him no ill will. Being pathetic, I let it stand like that and Richard and I lost contact for a few years.

World domination imminent

I was later driving though our home town and I saw Richard walking along. I stopped, wound the window down and Hallooed him. He Hallooed back, we became friends again and for the next few years our friendship entered into a beautiful halcyon period. This really was an Arcadian time for me, I had just got to university and felt a huge expansion in my horizons, and I spent a lot of my summers and many weekends back in my home town, in Richard's house, where we plotted world domination. Or, if we didn't feel like world domination on that day, we went to the pub and got drunk. I'm left with a wonderful sense of us two just feeling young, and like we were going to refashion the world how we wanted it. If you were to ask me what exactly it was that we thought we were going to do to dominate the world, I couldn't really give you specifics now. There's a half finished, half illustrated children's book we wrote in my loft somewhere. Its about a little girl who imagines her house is full of night time monsters, only to find out that actually it is. I'm not quite sure what the moral is in that, but it seemed funny at the time, and integral to our plan for world domination.

We were into our children's books then. We both loved Alice, and in defiance of critical sentiment, Sylvie and Bruno too. Richard created topiary chess pieces on his lawn, and I tried to convince myself that a detailed study of the Complete Works of Lewis Carroll would help me in my Advanced Logic exams.

Richard was working as a tree surgeon and I did summertime voluntary work for an inner city farm in central Birmingham. The sky was mostly blue and the world was stuffed with possibility. Richard had taught himself the guitar and I learnt the mandolin. Our repertoire consisted of the Irish folk dance Drowsy Maggie, the hornpipe The Trumpet (better known as the theme from Captain Pugwash) and nothing else. We would have dearly loved to play some Incredible String Band songs but we weren't good enough. I wasn't good enough. He was. He could play and sing at the same time, if adequately disinhibited by alcohol. Collectively though, we weren't good enough.

One very foggy Christmas Eve we went to the pub and both got so drunk we could hardly stand, then started the rather long and very uncertain journey home on foot. Out of the fog reared a very tacky plywood Christmas tree stuck in someone's front garden. Richard immediately took offence. He was a tree surgeon, and this was not a proper tree, but a tawdry imitation of one. He aimed a righteous kick at it, overbalanced and fell over backwards into the fog. I was confused, for where there was previously a Richard, now no Richard was there to be seen. I heard muffled swearing in the fog and he reappeared several moments later, having to satisfy himself with shaking his fist at the tree. We did get home that evening I'm sure, though I have no other memory of that Christmas.

Dear sir. It has come to my attention...

We also, in those pre-internet days, wrote letters to each other. Great big sheets of scribbles and drawings and poems. We had immediately settled into a secure pattern of not doing the obvious boring thing in letters, such as telling each other what we were doing, but concocting enormous screeds of unutterable nonsense with no bearing on reality at all. I was avidly reading P. G. Wodehouse at the time and I remember the joy I felt after I had sat with a pen and paper, let words wash out onto the paper, and found that what was on the paper was pretty respectable pastiche Woodhouse. Its how I still write now, given half a chance. Richard excelled at illustrating his letters and they were packed with cartoons. We both loved drawing cartoons and we happily traded off one another developing our styles in mutual appreciation. Richard then decided he was an oil painter, then a stop frame animator. Then one night I went round his house and he answered the door in an Oscar Wilde smoking jacket. I had lost track of what he was, or wanted to be by that point.

Wicker Man. Darkest Olton.Through all of this there was a central thread of Richardness though; he wasn't just a gadfly, picking up and discarding enthusiasms as he went along. Each one was an expression of who he was and made sense if you knew him. And then there was the mimicry. He could do a legendary James Mason impersonation. He was also an expert on black and white comedy films, golden era Hollywood, 1950's radio comedy, Syd Barrett, Land Rovers, DIY distillation, and so on, and so forth. There was also the annual Near Death Experience that was his bonfire party. Enormous wicker men were built in his garden, stuffed with fireworks and set alight, collapsing an hour or so later in a burnt out eruption of sparks, occasionally into the crowd of bemused friends invited round to watch who had to scatter pretty fast. The years when the wicker man didn't half kill someone were the boring ones.

So You Want to be a Cartoonist?

Spool forward a few years and he'd given up tree surgery and was regularly submitting cartoons to the Oldie, which got published, and to Private Eye, which didn't. Then apparently out of nowhere he won The Times Young Cartoonist of the Year Award, and I accompanied him to a swanky knees up of an awards ceremony at Simpsons in the Strand. I made a star struck idiot of myself fawning over Steve Bell.

He then was getting regularly published in several magazines, he joined the Cartoonists Club, which seemed to me to be little more than an excuse for monthly beer swilling in a pub near Fleet Street, and there was even talk of a regular job with The Daily Express, but that came to nothing. This went on for a few years and he learnt the tricks of the trade, developed his drawing style until it became very mature and assured. He was getting bored with it though. He carried on with The Cartoonist Club, becoming their secretary, but I think after a while that was the only association he had with being a cartoonist; he wasn't actually submitting any work for publication, just turning up for their monthly beer swill. He did meet Terry Parkes, the cartoonist better known as 'Larry' who drew the opening credit sequences for many of the Carry On films (which Richard loved) and they became firm friends for a year or so before Terry died. Richard was chuffed to bits to be able to name Terry as a friend.

The Viking long house in the garden was gone by this time, replaced a by a replica medieval barn he built from green wood that he'd collected as a tree surgeon. It really was a marvel; no nails, screws or glue in the construction, only held together by wooden pins. In it he built a forge, bellows and everything. Then he started tool making. Before that though he decided he was a cider maker. The garden suddenly filled with apple trees, but not your normal apple trees, oh no. Traditional English varieties of apple particularly suited to cider making, with lots of tannin and no sugar in them (or possibly lots of sugar and no tannin; I can't remember which way round it was. The apples tasted horrid, made incredible cider though.) I remember sleeping on the floor in his back room one time, being gently lulled to sleep by the sound of the regular glub of the air locks on the fermenting demijohns that lined the room.

Levelling The Land. Track One.

The thing is, through all this, there was only one way of doing things, and that was Richard's way. There was no other way. Even to the extent that he would deliberately and bloody-mindedly get it all wrong when forced to do something not in his way. This was appallingly frustrating for me who could see the enormous talent he had, but would not exploit or develop those talents unless he did so in his own way. He aversion to formal education that he developed at that stupid school of ours, held him back because he couldn't see that formal education was a way of bettering yourself; it was a way of those in authority clamping down on your creativity.

We used to go a folk festival, and he decided that you didn't go to festivals to do anything as trite and obvious as listen to the music, he went to absorb the spiritual festival vibrations. He would stake out the point on the field which he judged was the best point to absorb spiritual vibrations. Sitting anywhere else was untenable. The problem from my point of view was that this point was always just at the point at which you couldn't really see or hear the performers on the stage. I was trite and obvious enough to want to listen to the music. So I would trot off on my own to listen to the bands while he would absorb the spiritual vibrations somewhere off on the far horizon. One time I remember listening to hysterically funny Vin Garbutt who basically delivered brilliantly dry stand up comedy, occasionally punctuated by some songs. Richard would have loved him but he was too far away with his vibrations to hear. After the gig I stumbled back to find him and babbled about how wonderful Vin Garbutt was. I was ignored. We weren't there to listen to musicians.

A few years later I got a phone call to say that he'd just seen Vin Garbutt playing in a local folk club, and 'Wow wow wow! He's brilliant! You've got to hear him. He's playing again in a month. You've got to come see him. I've got you a ticket.' Yes, I explained, I know. I've known this for a long time. But there it was in a nutshell; I could lead him to water, but under no circumstance was Richard about to drink, or even admit that there was any water there. Only when he found it in his own way was it acceptable. So much of the stuff I wanted him to do or know about was met with this blank lack of acceptance. It drove me mad, but that was Richard.

After years of grumbling at me for being into those computer things, he suddenly decided to get one, hooked it up to the internet and sent me the following memorable email: 'My Dear Doctor, Screens full of naked women. Please advise.' One of the last conversations we had concerned his discovery on the internet of 'molecular gold' which when ingested apparently turns you into a god, filled with omniscient powers, overflowing with a joy that feels like 'a permanent orgasm'. I tried to suggest, in my trite and obvious way, that there was stuff on the internet that you find from time to time, which isn't entirely factually correct. He was having none of it. I wonder if he ever did get his molecular gold. If he had it might explain why we lost contact. I wouldn't bother phoning anyone if I was having a permanent orgasm. Would you?

The End

Richard Tomes and a wicker man (Richard is on the right)A few weeks ago I idly decided to Google him (which I did occasionally), and found a woodworking site which regretted to announce his death. He'd died two days before. Since we lost contact he'd concentrated on his tool making skills, making handcrafted wood planes in his back garden forge. There's a website out there showing his work. It is beautiful.

So that's that then. Plan for world domination didn't quite come together.

 

http://www.richardbutterworth.co.uk/blog/18-richard

Dear Internet… Really?

29th March 2014

Dad used to listen to BBC Radio 3 in the car. This was, I'd guess, the late 70s, early 80s. To my fresh young ears it was awful. Turgid, dull and incomprehensible. I remember things came to a head on one journey when some endless piece of ear bending serialism was delivered into the car with no introduction or context that I can remember.

'Oh, for heaven's sake.' said Dad and switched the radio off. That was the last I heard of Radio 3 for a quarter of a century.

My interest in classical music grew through my 20s, mostly due late night listening to a programme on the BBC World Service where I discovered the Brandenberg concertos, and other delights. I used to put Classic FM on in the office when I was doing my PhD, but eventually gave up, having been thoroughly patronised and insulted by their adverts. I did not want a golf holiday then, do not now, and hopefully never will. All this time Radio 3 remained a vague, unapproachable threat on the radio dial. A place, I thought, where music is revered with crinkled, very high brows and belittling sneers to those of us who didn't have much of a musical upbringing. A place not for me, nor the likes of me.

In late 2001 driving home from work, I decided that I could not face listening to the Radio 4 news any more; the War on Terrorism was too depressing. But where on the dial to go? Far too old for Radio 1, not interested in Radio 2, Radio 4 too depressing, local radio too vapid.

'Oh, for heaven's sake' I said, and pressed the previously untouched Radio 3 button.

The Golden Age

Over the next few months I was enraptured, not just by the music but by the completely unstuffy, approachable presenters who oozed passion and joy in the music they introduced. I made a list of the words routinely used on Radio 3 to described the music; 'glittering', 'revolutionary', 'spiritual', 'rousing', 'devastating'. I even shoe-horned this into one of my lectures on programming, asking why this sort of language is the preserve of the arts, not of science or engineering.

What a revelation to me that this music and this radio station were the most accepting, least elitist on the dial. With other genres of music you're either the wrong age, have the wrong attitude, the wrong culture or the wrong clothes. On Radio 3 there was not the slightest whiff of exclusion; all welcomed, a passion and curiosity about music, purely for music's sake was the only entry criteria. The door that I'd started to open was flung wide and I was hugely grateful. Over a decade later I still feel that I've only scratched the surface. I'll listen to that ear bending serialism now, because someone who knows it and understands it has explained it to me with care and passion. I still don't like it much, but maybe give it time. At least I now don't believe that that music is somehow inherently not for me.

To me the thing that unites Radio 3's output is the belief that whatever genre I'm listening to, it is pretty much the best there is; that it is music made purely for the love of the music, free of commercially driven compromise that diminishes most other forms of expression. After John Peel died there was much soul searching in the media, asking where was his replacement, where was the voice of pure enthusiasm for music, a voice willing to experiment and go in new directions, to invite the listener along for the ride. Even if the listener didn't like the places they were ridden to, that wasn't remotely the point, the point was that it wasn't boring. It was challenging. To me the answer to the question of where John Peel's replacement was is simple. Its Radio 3. And it is not just a few hours a week, tucked into the darker corners of the schedule, its all day, every day.

Oh, so very wrong

Then after a few years I was sitting bored in my office and started Googling around to see what others thought of Radio 3 and imagine my shock at finding that -- apparently -- I'd got it oh, so very wrong.

Take to the internet and you find that Radio 3 is completely dumbed down, a pale shadow of its former self. Filled with inane chatter and fatuous, smug presenters. Almost as bad as, in fact as bad as… and here it comes, the ultimate insult,  it is worse than Classic FM. Anyone who suggests otherwise is an idiot or a BBC stooge.

What amazed me about all this online venom is that classical music is supposed to be the ultimate enrichment to the human condition; music that challenges and questions us about what it is to be us, and that can answer many of those questions. I remember first hearing Pärt's Spiegel im Spiegel (on Radio 3 of course) while watching my daughter asleep in her cot and feeling like the music had clothed some essential part of me in sound. You would think that people who have spent all their lives being enriched by this wonderful music would be the most enlightened, delightful, erudite people you could meet.

But do you get that impression from reading Radio 3's Facebook page? Or do you rather get the impression that Radio 3 listeners are a dismal bunch of pedantic misanthropes, engaged in a battle with each other to come out with the most self righteously outraged comment, utterly lacking in humour, grace or any form of panache? Do you rather get the impression that they hate modern life, and in particular all those horrid young people who hang about the place, having fun and being enthusiastic about things?

So you go to the effort of liking Radio 3 on Facebook and then spend all your time making snide comments about it? Really? How much time do you think you have in this world, that you think this is a useful way of spending it?

You take to Facebook in a flurry of indignant pedantry to make some big issue about some minor mistake made during 'Building a Library' but you misspell 'library'? Really? I was wiping the irony off my laptop for weeks after that one.

You take to the internet to decry the lack of standards, discipline and respect in modern society, and then follow it up with deeply personal, offensive and childish attacks on individual presenters? Particularly the ones who who have the twin failings of being both young and female? None too preux chevalier, really.

And as for the hapless chap who took the time to castigate Radio 3 for playing Classic FM-ish lollipops like 'Short Ride in a Fast Machine' but got the name of the composer wrong. I mean really. Even I know that.

But then this is a completely unfair picture of Radio 3 listeners, and, believe it or not, this is absolutely not an attack on Radio 3 listeners, because all the ones I've met are all lovely in every way. I'm just talking about a noisy minority who think they own the art form and will hose abuse all over anyone who doesn't share that opinion.

Misanthropic, Ugly, Dreary Comments by Anonymous Trolls

Its not just Radio 3 listeners that seem to think that the internet is the perfect receptacle for their dreary misanthropy. As well as classical music, my other musical love is for folk music. But, oh my goodness, there's a website which proclaims itself to be the centre of all things folk, centred mostly around a discussion group. This discussion group is just the most wince-makingly dreary place. I've lost count of the number of times that I've been Googling this, that or the other about a folk singer or group, only to finish up on the site in question and spend the next half an hour, mouth hanging open in amazement, scrolling through page after page of snotty, narrow-minded grumbling.

And whatever you do, don't waste your life reading up on what they think folk music is and/or isn't. In summary, take the sentence 'folk music is what I say it is, and anyone who disagrees is an idiot' then repeat minor variations on this sentence several hundred times. There. I've read pages of tedious drivel, so you don't have to.

I sometimes think it's a generational thing, because the demographic for folk and classic music has an age profile that, shall we say, tends towards the autumn years. And maybe we all have a fixed amount of anger that needs venting, and lacking major global conflicts and things that actually matter, we chose to vent about trivialities on the internet. (As, you'll have noticed, I'm currently doing. There's paradoxical for you.) There are clearly all sorts of cognitive biases going on, particularly the reinforcement bias where you surround yourself with a clique of like minded misanthropes and finish up trying to outdo each other with who make the most miserable comment, completely losing any sense of how dreadful you appear to anyone outside your clique. If I was twenty years younger I'd very probably write a thesis about it.

Maybe I shouldn't worry about it. I realised a long time ago that I only read the folk music discussion group in order to get annoyed by the rubbish posted there, and vowed never to read it again. A vow I have kept, to no noticeable ill effect. I should probably do the same with the Radio 3 Facebook page, and just revel in the glory of listening to the radio station and not worry about the assorted piffle that pedants care to post about it.

And the answer is...

But before I go, if your default setting is to switch your computer on, navigate to your favourite discussion page and then spill some bile over it, I have the following questions...

Are you missing the fundamental irony that in the world wide web, you have been handed a means of communication undreamt of in human history, that just about all the information there has ever been is delivered to your laptop, that you can use to communicate all over the world with hardly any barriers and minuscule cost, and you use it to do what exactly? You use it to moan about how rubbish modern life is?

Really?

http://www.richardbutterworth.co.uk/blog/16-dear-internet

Varnish and Joomla!

17th November 2013

In the summer the BBC did a radio documentary about The Full English project. The interest in the site was such that the server couldn't cope, and was gummed up by the high volume of traffic for half an hour or so during and after the program.

Clearly not ideal.

Two months later the BBC repeated the documentary, and so we had to find a way to get the server to cope with the demand. We opted for Varnish -- a very highly spoken of HTTP cache -- which holds copies of popular pages in memory and serves them very efficiently from memory instead of getting them from databases and hard drives, which is much more inefficient.

The sites are Joomla! based. The problem is that Varnish and Joomla! don't fit together very well. Joomla! always passes a session cookie to the browser and the default behaviour of Varnish is to always go to the server for pages with cookies and bypass the cache; so by default Varnish will have no effect because the cache will always be bypassed. You could tell Varnish just to ignore cookies, but this also causes problems because if the user is logged in then the site should bypass the cache, but if the user is a guest, then its safe to server content from the cache.

There's disappointingly little advice and discussion to be had from Mr Google, the main contenders seeming to be massivescale's commercial product where Joomla! is patched to stop it generating the session cookie. I wasn't too keen and shelling out for massivescale's version, and most Joomla! experts advise against patching the core. (That said, I haven't tried massivescale's product, so certainly have no other comment to make about it -- they offer documentation and some support -- so if you're running an important site and are not so much of a programmer, probably best to go with them.)

There is also some likely looking code on the snipt site which recommends adding a little code to the template or one of the plug-ins to generate a response header which signifies when a user is logged in or not, as follows...

// Set user state in headers
if (!$user->guest) {
	JResponse::setHeader('X-Logged-In', 'True', true);
} else {
	JResponse::setHeader('X-Logged-In', 'False', true);
}

Varnish can then read the header and serve content from the cache or server appropriately.

So we went with the configuration from the snipt site, and Ta Da! with a bit of minor tweaking the cache worked and the sites survived the repeat of the BBC documentary, with higher traffic than the first airing. So relief and happiness all round.

Then the problems started creeping in. Mostly to do with logging in and out. We haven't got many users logging in; currently only a handful of volunteers who are submitting crowd sourced content. When a user went to the log in page and logged in, then they would be shown a cached version of the log in page (ie showing them not logged in). A few refreshes and maybe jumping to the home page would -- usually -- get the response header working, the cache bypassed and everything working. This was clearly not ideal though, as we were having to tell our volunteers to bang refresh a few times and hope for the best.

The problem is with the response header, because the header comes from the server in the response, not from the client. So if Varnish decides to serve from the cache as opposed to from the server, you can set as many response headers as you like; Varnish won't see them because its serving a cached version, and if it has to go to the server every time to find out whether the user is logged in or not then that defeats the point of the cache.

The signal that user is logged in must come from the browser -- via a cookie -- not from the server via response headers.

What would be ideal if there was some way of examining Joomla's session cookie to see if it is a guest session or not. But as far as I can see there isn't, and there are probably security related reasons why. So we implemented another cookie to run in parallel with the Joomla session cookie which can be easily detected by Varnish's VCL and is simply a flag telling Varnish to bypass the cache or not. So we can tell Varnish to bypass the cache not only when the user is not a guest, but also when the user is doing something with log in pages (ie. when the Joomla component is 'com_users')

So, firstly there is a Joomla plug-in to set/unset the cookie, as follows...

class  plgSystemVarnish extends JPlugin
{
	function onAfterRoute( )
	{
		$user = JFactory::getUser( );
		$option = JRequest::getVar( 'option' );
		if( !$user->guest || $option == 'com_users' )
		{
			setcookie( 'varnishbypass', 'True', time( ) + 604800, '/' );
		}
		else
		{
			setcookie( 'varnishbypass', '', time( ) - 604800, '/' );
		}
	}
	// setting the cookie in the afterRoute event is probably adequate,
	// but in an outburst of belt and braces I decided to set/unset the
	// cookie on the log in and out events too. Probably unnecessary,
	// but apparently harmless.
	function onUserLogin( $response )
	{
		setcookie( 'varnishbypass', 'True', time( ) + 604800, '/' );
	}
	function onUserLogout( $response )
	{
		setcookie( 'varnishbypass', '', time( ) - 604800, '/' );
	}
}

...then an adaptation to the VCL configuration from the snipt site as follows...

	if( req.url ~ "^/administrator" || req.request == "POST" ||
				req.http.cookie ~ "varnishbypass" )
	{
		return( pass );
	}

...and even more Ta Da! this works and respects the users logging in an out.

Two observations, though. Firstly this works for our site, but as I said, we only have a few people logging in and out, and they're volunteers who are fairly tolerant of the site doing odd things; they are not members of the public who would rightly give up in disgust and walk away if the site was doing odd things. I'm making no claims for the general applicability of this method, it is entirely possible that this solution will introduce strange behaviour if your site does log ins differently to our sites. But as far as I can see you have to signal that a user is logged in or not via a cookie, not by headers. Secondly it was a real mare getting this working on a live site; if at all possible you should design your site to work with Varnish from the start rather than adding it in later. 

 

 

http://www.richardbutterworth.co.uk/blog/15-varnish-and-joomla

I did a PhD and did NOT go mad (2013 update)

2nd November 2013

In about 1997 I was asked to give a presentation in (what was then) the School of Computer Science at Middlesex University about my experience of doing a PhD. It was to be given as part of an induction day for to the PhD students just starting off in the school.

I'd just completed my PhD and felt somewhat jaundiced about the whole process, mixed with the startling feeling of exhilaration that lasts for several months after completion and decided that I should tell the new intake exactly what my experience had been -- warts, spots, disastrous relationships, and all.

This was probably somewhat unfair on the audience.

 It was also probably a bit unfair on the senior academics who had spent the morning telling the new intake how this was going to be the best time of their lives -- the thrill of pushing the boundaries of knowledge -- the freedom to pursue challenging research paths. My subsequent talk came as a contrast.

I'd done some cartoons to go on acetates for the overhead projector (remember those, academics over 40?) and instead of doing the usual thing of stuffing the cartoons in my cartoon drawer never to see the light of day again I decided to write up the text of talk, scan the cartoons and stick them on one of those new fangled web pages. This was long before the days that social media had been dreamt up, or even before the days when comment boxes were common.

In the subsequent sixteen years the web page travelled with me in the various incarnations of my different personal web sites as my career hopped about here and there, eventually finishing up forgotten about on some old server space that a friend had lent me when I first set up a website outside the groves of academe. I'd been aware that a few people were looking that page, and judging by occasional email I got saying from distraught students saying how I'd described their wretched lives perfectly, its contents seemed to be still relevant.

It came as quite a surprise when I finally started taking the strange world of Twitter seriously to find out that not only was I did a PhD and did NOT go mad still online but seemed to have gathered quite a loyal following. After a bit of Googling I was also rather gratified to find that my words had taken on quite a life of their own, and in the minds of several internet authors at least, I had been promoted to Professor Butterworth.

So as I'm setting up my new website and blog I thought it was obviously time to drag I did a PhD off its forgotten server, into the 21st century, and into the social media world.

It is, I'm pretty sure, the only thing I wrote as an academic that was read by more than ten people. A lot has changed since I wrote it, but the crucial importance of chocolate has not. And never will.

http://www.richardbutterworth.co.uk/blog/12-i-did-a-phd-2013

I did a PhD and did NOT go mad

3rd November 1997

Before reading these wise words advising you how to do a PhD (inspired by three years of the author carefully and diligently banging his head on a table) you are requested to read and digest the following irony...

The only way to find out how to do a PhD is to do one. Therefore all advice is useless.

To say that I enjoyed doing my PhD would be a lie, not just an ordinary lie mind you. More the sort of lie one would normally associate with Tory party conferences. A big wobbly lie with a dusting of sugar on top. At times I hated my PhD, so why do I have any authority to give advice on doing a PhD? Well, I don't claim to have any -- other than the fact that I completed and passed the thing, so I must have done somthing right.

This page is an attempt to...

There's only one way of doing a PhD and that's your own, that's your own, that's your own...

A PhD, by its very nature, is a very individualistic venture. There is no right way to do a PhD (there are however a multitude of wrong ways). This is the first big surprise for people who are starting their PhD having completed their undergraduate degrees -- there are well defined correct ways of getting a degree (usually 'turn up to lectures, do course work, revise for exams, use a modicum of common sense') but such prescriptive techniques don't work for research degrees. The award of a research degree effectively says `This person knows how to do research in his/her chosen area' and `research' is a nebulous, difficult to nail down thing which relies on insight, lateral thinking, inspiration and a lot of hard work. An undergraduate degree is a lot of hard work, but doesn't put so much emphasis on inspiration. Most (if not all) people cannot sit down and say 'right, today I'm going to have some inspiration'. The unpredicable nature of progress in a PhD means you spend a lot of time not sticking to the deadlines you set yourself. This is dispiriting.

In most departments there's always one smartarse who loudly proclaims that doing a PhD is easy and he (its usually a he) can't see what all the fuss is about, and he's just written another three chapters this morning, and he's submitted another five journal articles. People like this are lying, showing off, from Mars or over-compensating for inadequacies in other areas of their lives (if-you-know-what-I-mean). Ignore them.

The loneliness of the long distance researcher

The next big surprise for people who are starting PhDs after an undergraduate degree is just how excrutiatingly lonely a PhD is. When you submit your thesis you have to sign a piece of paper that says 'This is my work, my work alone, nobody else's, all mine, nobody but me did it. Honest.' A PhDs is so narrow and focused that the chances are that you and only you is going to understand anything about it. This is known as the Loneliness of the Long Distance Researcher and you have to get used to it. It means you're going to spend at least three years wandering around with a great chunk of bizarre, irrelevent nonsense in your head that only you can relate to. Get used to people's eyes glazing over and them shuffling in their seats when you try to explain exactly what it is you're doing. Try to avoid emotional entanglements with people who say `penny for your thoughts' during romantic moments. If, like me, you say `the problem of formally refining liveness properties stated in a temporal logic for reactive systems' then you're likely to find yourself rapidly emotionally unentangled. Or at least they ask for a full refund on their penny with a written apology. Anyway, quite frankly, after a couple years of doing my PhD the last people I wanted to spend any time with were the sort of people who would be remotely interested in my work. To quote Groucho Marx 'I wouldn't want to join a club that would have me as a member.'

Many departments have well integrated research programs with seminars and meetings and other such social gatherings. This is A Good Thing and will go a long way in relieving the loneliness of the long distance researcher. Many departments are, however, abysmal in this respect. If you're an active go-getting sort of person you may try sorting out social gatherings but be prepared for knockbacks. I did my best to get my colleagues down the pub, but the world record for number of research students in one pub at the same time was (from memory) nine out of forty -- and that was only when two Christmas parties accidentally went in the same pub at the same time. Remember, if you can't persuade other researchers to leave the sanctity of their computer screens and come down the pub its probably not your fault -- they've probably forgotten what a pub is, what you do in one and why. When you start your PhD you'll probably think that they're a sad bunch of herberts, but believe me, by the time you've spent three years doing a PhD you'll have a lot more sympathy for them.

Thank you for your support. I shall always wear it.

Although doing a PhD is an individual thing your department should give you as much support as possible. I fear that the attitude of some departments is rather Victorian to their research students -- that having a miserable time when doing your PhD is character forming and uplifting and you'll thank them for forcing you to spend three years in an unheated office with only a wasp nest for company in the long run. Such sentiments are usually followed by `anyway when I did a PhD we didn't have offices -- we did our research in the departmental lavatories and were hosed down once a month by the cleaning staff. You have it lucky'. Anyone of this attitude really deserves a kick in the pants of the most severe measure, but I bet you'll come across at least four of them. If you get treated badly by your department then don't be shy -- moan. Don't moan for the sake of it because you'll just get peoples backs up, but make sure that those in authority know about your grievances and don't have the excuse once people get fed up and leave their degrees halfway through (as they do) that it wasn't known how upset they were. It is likely that you have a good first degree and could therefore be In The Real World doing a job that pays about ten times as much as the pittance you get as a stipend for a research student. Really you're doing your department a favour by publishing results and upping their research credibility and hence they should show you at least a little respect and common decency.

Use your hanky

There are going to be times when it all gets to you and you can't cope any more. (The day I found a paper that I'd missed in my literature survey that covered all the `new' stuff I'd done in the previous nine months was my own personal nadir.) What do you do in such circumstances?

Cry.

Your eyes out.

Since the tragic death of the Princess of Wales we all now know how to express our negative emotions fulsomely and publicly without caring a jot. So forget your stiff upper lip and let it tremble. But don't waste that outpouring of emotion -- make it count as much as possible. In a sentence, burst into tears in your supervisor's office. If you're really determined to gain dramatic effect then wait until he/she's entertaining some important guests -- Profs from other universities or co-authors on important papers. However if your supervisor is one of those 'My door's always open' types who spend all year at conferences in the Yemen or can't be seen without booking two months in advance with his secretary then you may have problems. An email isn't very effective in this case...

To: Prof Vacation [vacate@never.here.ac.uk]
From: Hopeless inadequate useless bag of rubbish [self_pity@here.now.ac.uk]
Subject: Bleeeeeaaaah!

Dear Prof,

Boooooohoooooo, snivel, snivel, bleeeeaaaaaah...

/END MESSAGE

Its not the same is it?

While we're on the subject of dealing with emotional catasphrophes I should mention the crucial role that chocolate played (and is still playing) in my academic career. There is no problem known to science that cannot be cured by the liberal application of chocolate. Leading doctors have testified to its mystical curative powers -- it is known to contain all sorts of wholesome, bracing chemicals that get straight to the happiness centres of your brain and get them working at full pitch, scattering love and joy and contentment all through your cortex. Leading dieticians and skin care specialists may make the odd carping comment, but take no notice. Just consider the following scientific breakthroughs directly attributed to chocolate...

Need I say more?

In case of emergency...

There are of course several methods of stress relief to be tried -- popular ones include Staring Out of Windows and Watching Old Black and White Films. Obviously with minimum effort you can combine these with Eating Chocolate for a previously unprecedented amount of stress relief. It is more difficult to combine Staring Out of Windows and Watching Old Black and White Films though, at least not unless you put your telly outside your window. (Not recommended during inclement weather or for those in high rise buildings.) I have been advised on medical and moral grounds not to include Getting Monsterously Bladdered and Uncomplicated Sex in my list of stress relievers, but you'd probably thought of those already.

In order to not become entirely divorced from reality it is a good idea to get a hobby and the less your hobby has to do with the subject of your PhD the better. If you are considering doing a PhD on a subject you really enjoy and find fascinating then beware -- the chances are that after three or so years of studying nothing but your chosen topic you'll hate the very mention of its name. Do you really want this? My PhD was in a Bizarre Corner of Computer Science and during my PhD some of the hobbies I'm willing to admit to are learning to play the mandolin, taking watercolour lessons, going on some very long walks and sewing. My chocolate consumption skills also improved dramatically as I think I may have mentioned elsewhere.

Its the end of the world as we know it, and I feel fine (sort of)

So, in summary...

Further reading...

If you're really determined to keep wasting time on the internet instead of knuckling down and doing that really dismally boring data analysis that you set yourself a deadline to complete five weeks ago, you can have a look at my notes about I did a PhD and did NOT go mad. Then after that there's Facebook and Twitter, and your own blog to update...

 

http://www.richardbutterworth.co.uk/blog/13-i-did-a-phd