Scott Alan Henry

Share what you think you know…

Amazon S3 Database Backups

NOTE: Once this is done the first time you can just copy the script and s3/ to the new site directory and modify the files.

1
2
3
4
5
6
cd /opt
sudo wget https://github.com/s3tools/s3cmd/archive/master.zip
sudo unzip master.zip
cd s3cmd-master/
sudo python setup.py install
which s3cmd (/usr/local/bin/s3cmd)

Just above Drupal root:

1
2
mkdir s3; cd s3;
nano s3/mysql.cnf

Add this to the mysql.cnf file:

1
2
3
[client]
user=‘DB_USER'
password=‘DB_PASSWORD'

Make a log directory:

1
mkdir log

You may need the following:

1
sudo apt-get install python-dateutil python-magic;

Now we can configure s3

1
sudo s3cmd —configure;

(You need the Access Key and the Secret Key from s3.) You should see output like: Configuration saved to ‘/home/[USER]/.s3cfg’ Move the .s3cfg file to your s3 directory

Add the script just above Drupal root:

1
nano sqlbackup-s3.sh;

Add the following contents and make changes to variables where necessary:

If sendmail doesn’t send, make sure the log file has a subject line. Should be added by the script:

1
subject: S3 database backup

Run the script and make sure the db uploaded to S3.

Now we can setup a cron job.

In the CLI:

1
$ crontab -e

Add the following replacing the path to the script:

1
2
# Backup the database at 12am and 12pm everyday.
0 0,12 * * * /PATH/TO/DIR/ABOVE/ROOT/sqlbackup-s3.sh

Help – SA-CORE-2014-005 Hack

This might at least help find malicious code that could have been injected in module files. Does a cool side-by-side diff and exports to a file so you can easily move through the file and see any differences.

In the following command make sure to put the ‘good’ drupal install before the potentially ‘bad’ or existing install.

In this case ‘drupal’ is just a

1
git clone --branch 7.x http://git.drupal.org/project/drupal.git

and a

1
git checkout 7.32

and ‘pre-update’ is my old drupal install that I want to check for malicious code.

In this case the “drupal” directory will be on the left. Just switch them around to see the other directory on the left. I chose to do it this way because I wanted to see everything for the the “good” code and see just lines that were different in what I had before the update.

1
diff -r -y --left-column drupal pre-update  > dirdiff.txt

This does a recursive (-r) diff and places the results side-by-side (-y) in a file. The (—left-column) makes it easier to spot the diff.

On mac you can just hold down fn + (down arrow)

You should at least see:

This is the patch that was posted for Drupal core: https://www.drupal.org/files/issues/SA-CORE-2014-005-D7.patch

Also

Something else you can do for any code you may have been version controlling (we all version control our custom code, right?):

I looked back through my git branches, as I tend to keep them for quite sometime, and I found a branch that I had labeled updates-10-13-14 so I know that these are any updates I may have pushed on the 13th of October. This is two days before the ‘catastrophe’ so this code shouldn’t be affected. I’m working with the Panopoly distro as my base install so all of my custom code exists in sites/all and that’s what I push to GitHub. In the sites directory I placed a directory called: old_all. So here is what I did:

From your sites directory:

1
2
3
4
5
6
mkdir old_all
cd old_all
git init
git remote add origin git@github.com:scottalan/[NAME_OF_REPO].git
git fetch origin
git checkout origin/updates-10-13-14

Now I have my PRE ‘possible hack’ code in old_all and where ever I am now in the original all directory. Now it’s just a matter of running the diff command.

1
diff -r -y --left-column old_all all  > all-diff.txt

You could also exclude certain files to give you less to look through:

1
diff -r -y --exclude=*.css --left-column old_all all  > all-diff.txt

This would exclude any CSS files for example.

Now I can see a side by side diff of the changes. Of course you will see things that you recognize changing but keep an eye out for anything that doesn’t seem quite right.

Sure this is tedious but a heck of a lot better then trying to look through each file individually.

Good Luck! Don’t forget to checkout: https://www.drupal.org/drupalsa05FAQ

-Scott

Seldom Used

Thought I’d store some commands, that I find myself searching for, that I may not use all the time but would be nice to have in one spot.

Untar directory contents in the current directory.

tar --strip-components=1 -zxf file.tar.gz

OSX 10.8: Adding PHP Versions for IDE Development

You will need Xcode and Command Line Tools installed.

You don’t have to use Fink as your package manager but I find it nice to be able to just use the commands I am used to when working on a server.

Download the version of Fink you need for your OS http://www.finkproject.org/download/index.php?phpLang=en

Allow Fink to install to its default location /sw

I will assume you can handle the above so we will jump right in…

mkdir -p /opt/src

Found this snippet that I started with to handle the first part of getting everything into place, but had to alter it a bit so that it would work for my use-case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
#!/bin/bash

if [ $# -ne 1 ]; then
  echo "Usage: $0 expects at least one argument: [php version]"
  echo "i.e., 5.3.28 - http://php.net/downloads.php"
  exit
fi

PHP_TARGET_VERSION=${1}

PHP_INSTALL_PREFIX=/opt/phps

mkdir -p /tmp/src/php-${PHP_TARGET_VERSION}

cd /tmp/src/php-${PHP_TARGET_VERSION}

curl -# -L http://us1.php.net/get/php-${PHP_TARGET_VERSION}.tar.gz/from/this/mirror | tar -xz --strip 1

echo "Target Version: ${PHP_TARGET_VERSION}"
echo "Install Prefix: ${PHP_INSTALL_PREFIX}"

./configure \
            --sysconfdir=/private/etc \
            --prefix=${PHP_INSTALL_PREFIX}/${PHP_TARGET_VERSION} \
            --with-config-file-path=${PHP_INSTALL_PREFIX}/${PHP_TARGET_VERSION}/etc \
            --mandir=${PHP_INSTALL_PREFIX}/${PHP_TARGET_VERSION}/share/man \
            --infodir=${PHP_INSTALL_PREFIX}/${PHP_TARGET_VERSION}/share/info \
            --enable-debug

make

sudo make install

# Additional flags that can be set.

#            --with-apxs2=/usr/sbin/apxs \
#            --enable-cli \
#            --enable-debug \
#            --with-libxml-dir=/usr \
#            --with-openssl=/usr \
#            --with-kerberos=/usr \
#            --with-zlib=/usr \
#            --enable-bcmath \
#            --with-bz2=/usr \
#            --enable-calendar \
#            --disable-cgi \
#            --with-curl=/usr \
#            --enable-dba \
#            --enable-exif \
#            --enable-ftp \
#            --with-gd \
#            --with-freetype-dir=/sw \
#            --with-jpeg-dir=/sw \
#            --with-png-dir=/sw \
#            --enable-gd-native-ttf \
#            --with-icu-dir=/usr \
#            --with-iodbc=/usr \
#            --with-ldap=/usr \
#            --with-ldap-sasl=/usr \
#            --with-libedit=/usr \
#            --enable-mbstring \
#            --enable-mbregex \
#            --with-mysql=mysqlnd \
#            --with-mysqli=mysqlnd \
#            --without-pear \
#            --with-pdo-mysql=mysqlnd \
#            --with-mysql-sock=/var/mysql/mysql.sock \
#            --with-readline=/usr \
#            --enable-shmop \
#            --with-snmp=/usr \
#            --enable-soap \
#            --enable-sockets \
#            --enable-sysvmsg \
#            --enable-sysvsem \
#            --enable-sysvshm \
#            --with-tidy \
#            --enable-wddx \
#            --with-xmlrpc \
#            --with-iconv-dir=/usr \
#            --with-xsl=/usr \
#            --enable-zip \
#            --with-pcre-regex=/usr \
#            --with-pgsql=/usr \
#            --with-pdo-pgsql=/usr \

You can then run: sudo php-configure.sh 5.5.11 This assumes you have added the script to your $PATH.

Download this: https://github.com/wilmoore/php-version

Then you can export your path:

export PHP_VERSIONS="/opt/phps"

Type: which php and you should see you are using the default: /usr/bin

This gets you, at a basic level, the versions of php you may need for your IDE.

In your IDE you can set the interpreter to: /opt/phps/5.5.11/bin for example…

Translations in Drupal7

Trials and tribulations of a Drupal 7 multilingual site… This is way less complicated if you set up your languages from the start.

Assumptions:

You have a working Drupal 7 site. (I’m using 7.22) You know how to install modules. Some drush experience is good but not necessary. You need a second language on your site. I spent some time investigating the best options for creating a multilingual site and this is what I’ve come up with. This works for me and may not work for everyone. I will list the modules and versions that I’m using in hopes that it will help someone else.

There may be a few modules that you will need depending on your use case. For this post I’m only going to touch on the translation/multilingual modules that you need.

NOTE: I ran into this problem and thought I’d bring it up now as it caused some issues and quite some time trying to find a solution.

If you have existing nodes more than likely they have a language of ‘und ’ in the database. Please make sure you have a database backup before trying this. This is always good practice, especially if this is a production site. I recommend never running anything on production site or it’s database before first trying it out in a development environment first. Run this query in an update hook or (because I have a dev site) I just ran this query in SequelPro. UPDATE node SET language = ‘en’ WHERE language = ‘und’; ^ This assumes you are using English as you default language. ​If at any time something doesn’t seem to be showing up or displaying properly just clear the cache (drush cc all). I’m running Nginx, php5-fpm, and Varnish so I’ve had to do this a few times. This will just depend on your setup.

To get started you will need to download the following:

https://drupal.org/project/i18n https://drupal.org/project/l10n_client https://drupal.org/project/l10n_update https://drupal.org/project/tmgmt https://drupal.org/project/tmgmt_microsoft (for translations) https://drupal.org/project/lang_dropdown https://drupal.org/project/languageicons (adds icons to the switcher) https://drupal.org/project/title https://drupal.org/project/variable Modules that we will enable:

Core Locale (locale) Contrib Entity API (entity) Views (views) Ctools (ctools) Views Bulk Operations (views_bulk_operations) Internationalization (i18n) Entity Translation (entity_translation)

Localization client (l10n_client) Localization update (l10n_update)

Translation Management Core (tmgmt) Translation Management UI (tmgmt_ui) Entity Source (tmgmt_entity) Entity Source User Interface (tmgmt_entity_ui) Translation Management Field (tmgmt_field) Microsoft Translator (tmgmt_microsoft)

Title (title) Variable (variable) Variable Admin (variable_admin) Drush command to install all modules we need:

For this example I’m going to add Spanish as a second language. I’m also not going to use core’s Content translation module but rather going to do this using Entity translation instead. This will allow us to have one node with multiple translations per field vs. having to have multiple nodes, one for each translation.

Add another language:

/admin/config/regional/language Click + Add language Choose Spanish from the select list Click on the Detection and Selection tab (fill this in) From here you can go to: /admin/config/regional/translate and see how much has already been translated for you.

Entity translation:

/admin/config/regional/entity_translation For now we will leave the defaults selected but we will be coming back here to add Taxonomy term and Comment to the Translatable Entity Types. You should see that Node is already selected, if not then select it now and save. i18n:

/admin/config/regional/i18n Select Enabled languages only Tmgmt:

/admin/config/regional/tmgmt You can see that there haven’t been any jobs created yet. You may also see a warning at the top of the page: Entity translation is not enabled for any of existing content types. To use this functionality go to Content types administration and enable entity translation for desired content types. We will get to this in a moment…

Click on the Entity Sources tab We haven’t added any entities yet Click on the Translators tab We will be adding a translator as well. On the Settings tab we will be leaving the defaults. For all of your existing content types or any new ones:

/admin/structure/types Click Edit on the content type you want to add Multilingual support for Click on Publishing options and under Multilingual support select: Enabled, with field translation I also choose Hide content translation links because I will be using the language switcher block. Save

Now head back over to: /admin/config/regional/tmgmt/entity

You should see a list of nodes on your system that belong to the content type you just edited.

If you would like to use the Microsoft translator then follow these steps:

Enable the Microsoft Translator (tmgmt_microsoft) Now when you visit /admin/config/regional/tmgmt/translators you should see it. We need to configure the translator Click Edit You will need to head over to: https://datamarket.azure.com/dataset/1899a118-d202-492c-aa16-ba21c33c06cb and sign up for a free account. Under MICROSOFT TRANSLATOR PLUGIN SETTINGS​ Enter your Microsoft Customer Id and Primary Account Key

Just to make sure double check the REMOTE LANGUAGES MAPPINGS and make sure they are what you expect.

Save

There are a couple more things we need to do to make sure we make all of our fields available to the translator:

Go back to the content type that we are adding language options to. Under the Manage Fields tab you should see an operation under ‘Title’ that allows you to replace it. Do this. Now edit the ‘Body’ At the bottom of the page under Field translation click Enable translation Leave the migration setting checked I cleared cache here (drush cc all) Go back to the Entity Sources tab

Lets try a translation… Select one of your nodes and then click Request translation

You should notice that it wants to translate from English to Spanish and it should give you a word count.

Click Send to translator

When it’s done you will be directed back to the Entity Sources tab. Click the yellow Ready for review box next to your node. If it looks good (or like me, you think it looks good) then click the checkmarks to the right of each field and Save as completed. You should now notice that the yellow box is now green.

Add the Language switcher block to a region and then test it out and make sure you can switch back and forth and see the expected results.

Remember, you will only see Spanish for the nodes that you have already translated.

If you end up with this error: Rejected by Microsoft Translator: TranslateApiExceptionMethod: Translate()Message: Cannot find an active Azure Market Place Translator Subscription associated with the request credentials.message id=3817.V2_Rest.Translate.3361868A

Do this:

https://datamarket.azure.com/developer/applications/register and register your application. Use the clientid and secret you created when you created your account.