This post is an extension to the previous post I did explaining how to connect Twilio to Google Sheets and Google Data Studio , the missing part in that post was how to designate different phone numbers to different mediums or sources of a website traffic, and this is what I will be covering in this post.
Step 1:
Decide what mediums or sources you want to track, the most basic tracking if you have a low budget is tracking three mediums: organic, paid and others, a more advanced tracking will be:
Organic Google
Organic Bing
Paid Google
Paid Bing
Referral
Direct
You can also consider tracking email campaigns and offline campaigns like radio. Obviously the more mediums/sources you want to track the more phone numbers you need.
Step 2:
Assuming you are going after a basic tracking (organic, paid and others) you need to buy three phone numbers, I will assume they are:
111-111-1111 for organic traffic
222-222-2222 for paid traffic
333-333-3333 for others
Make sure to redirect those numbers to your own phone number, I will call that the default phone number and it is 000-000-0000
Step 3:
We need to save the source, medium and term for every website visitor in a cookie, you can do that using UTMZ-replicator just add it to your website using Google Tag Manger as a custom HTML tag and let it fire on every pageview.
Step 4:
Make sure the phone number on your website is contained in a unique class, something like below:
Add the JavaScript code below as a tag in Google Tag Manager and let if fire after the UTMZ tag in step 3:
....
var findnum = [
{ "number": "111-111-1111", "medium": "organic" },
{ "number": "222-222-2222", "medium": "cpc" },
{ "number": "333-333-3333", "medium": "others" }
];
function searchMD(findnum, medium){
for(var i= 0, L= findnum.length; i<L; i++){ if(findnum[i].medium=== medium) return findnum[i].number; } return ''; } var ga_source = ''; var ga_campaign = ''; var ga_medium = ''; var ga_term = ''; var ga_content = ''; var gc = ''; var c_name = "__utmzz"; if (document.cookie.length>0){
c_start=document.cookie.indexOf(c_name + "=");
if (c_start!=-1){
c_start=c_start + c_name.length+1;
c_end=document.cookie.indexOf(";",c_start);
if (c_end==-1) c_end=document.cookie.length;
gc = unescape(document.cookie.substring(c_start,c_end));
}
}
if(gc != ""){
var y = gc.split('|');
for(i=0; i<y.length; i++){ if(y[i].indexOf('utmcsr=') >= 0) ga_source = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmccn=') >= 0) ga_campaign = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmcmd=') >= 0) ga_medium = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmctr=') >= 0) ga_term = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmcct=') >= 0) ga_content = y[i].substring(y[i].indexOf('=')+1);
}
}
if (!((ga_medium =="organic") || (ga_medium =="cpc"))) ga_medium = "others" ;
document.getElementsByClassName('default-phone-number')[0].innerHTML= searchMD (findnum,ga_medium);
document.getElementsByClassName('default-phone-number')[0].href= "tel:"+searchMD (findnum,ga_medium);
...
At this point you can be sure that phone number will fire only based on their correspondent medium, at this point you can start building your dashboard following the post here
Call tracking is a vital service for any marketer, and when it comes to call tracking, Twilio is the largest telephony infra structure provider in the market that can help a lot with that; however, Twilio is geared more towards developers, nontechnical users can not do much using Twilio and they have to go for a plug and pay services like, Dialogue Tech and Call rail.
In this post I will provide Twilio's nontechnical users with a solution to create a friendly dashboard using Google Sheets, which could be eventually used with Google Data Studio for better visualization.
Step 1:
Make sure you have a Twilio account with at least one active phone number that redirects to your own phone number, Twilio has a nice guide here that explains how to find and buy a phone number, after buying the phone number you can use Twilio's built in web-hook to redirect it to your own phone number (see below)
Just add your phone number to the end of the web-hook's URL: http://twimlets.com/forward?PhoneNumber=111-111-1111
You can buy multiple phone numbers and use each of them to track a different medium, like organic, paid, and others. In this tutorial you can find how to rotate phone numbers based on incoming traffic to your website, if you are using the the numbers on dedicated landing pages or other mediums like off-line, number's rotation will not be required as each number will be served only for one medium.
Step 2:
Find your Twilio's API credentials by clicking the gear icon at the top >> settings >> copy ACCOUNT SID and AUTH TOKEN
Step 3:
Replace your ACCOUNT SID and AUTH TOKEN in the code below that we will be using in Google Sheets:
function myFunction() {
var ACCOUNT_SID = "*********************************";
var ACCOUNT_TOKEN = "*********************************";
var findmedium = [
{ "number": "+1**********", "medium": "organic" },
{ "number": "+1**********", "medium": "cpc" },
{ "number": "+1**********", "medium": "others" }
];
// you do not need to edit anything below this line
function searchMD(findmedium, num){
for(var i= 0, L= findmedium.length; i<L; i++){
if(findmedium[i].number=== num) return findmedium[i].medium;
}
return '';
}
var numberToRetrieve = 10;
var hoursOffset = 0;
var options = {
"method" : "get"
};
options.headers = {
"Authorization" : "Basic " + Utilities.base64Encode(ACCOUNT_SID + ":" + ACCOUNT_TOKEN)
};
var url="https://api.twilio.com/2010-04-01/Accounts/" + ACCOUNT_SID + "/Calls.json?PageSize=" + numberToRetrieve;
var response = UrlFetchApp.fetch(url,options);
var theSheet = SpreadsheetApp.getActiveSheet();
function search(xyz) {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
for(var i=2;i<12;i++){
var value = sheet.getRange(i, 6).getValue();
if(value == xyz){
var match = 1 ;
}
}
return match ;
}
var startColumn = 1;
var theRow = 2 ;
var dataAll = JSON.parse(response.getContentText());
for (i = dataAll.calls.length -1 ; i >= 0 ; i--) {
var found = search (dataAll.calls[i].sid) ;
var calldirection = dataAll.calls[i].direction ;
var midnight = new Date();
midnight.setHours(0, 0, 0, 0);
rowDate = dataAll.calls[i].date_created;
var theDate = new Date (rowDate);
theDate.setHours(theDate.getHours()+hoursOffset);
if (!found && calldirection == "inbound") {
theSheet.insertRowAfter(1);
If you are using multiple phone numbers for different mediums replace the values in the findmedium array, otherwise just leave the code as is.
Step 4:
Create a new sheet in Google Sheets and name the first 6 columns in the top row as below (feel free to change the names to any other names keeping the same order)
Step 5:
In Google Sheet go tools >> script editor >> delete any existing codes and replace them with the codes from step 3
Step 6:
In Google Sheets script editor click on the clock icon to create a trigger >> Click + Add Trigger (bottom right corner) >> Time-driven >> Hour timer >> Every hour >> save
Once you do that, in few hours the data will start to show up in your sheet as below:
Having the call date in Excel will give you a lot of flexibility to manipulate and extract data, for more visualization read step 7 which will explain how to create a simple report in Google Data Studio.
Step 7:
Connect the sheet you created to Google Data Studio by going blank report >> Create data source >> Create new data source >> Google Sheets >> Choose the spread sheet created in step 4 >> Click connect
You should be able to see an empty Google Data Studio report with a grid, now Insert >> Time series >> Add to report
If everything works fine you should be able to see a report like below (a daily total calls duration report) :
There is a lot you can do to customize this report but I am not going to go through all of that, watch this video here to learn more.
Would not be great if marketers can see Google Analytics traffic source dimensions in Salesforce? Unfortunately Salesforce by default doesn't provide any attributions for web leads pushed to Salesforce, I am not sure why considering that is not difficult to do that for a lead that is pushed using web-to-lead form where a simple additional code can push the lead source dimensions like source/medium and more other dimensions that marketers badly need to calculate ROI. This guide will provide a step-by-step method explaining how to push the lead source dimensions mainly source, medium and keyword (if available) to Salesforce, I will be using the same terminology used by Google Analytics to keep things consistent for marketers.
Step 1:
We need first to create few extra fields in Salesforce to save the lead source dimensions, in sales force go Salesforce > Set up > Object Manager > Lead > Fields & Relationships > New > Text
We need to repeat the process above few times depending on the number of diminutions we are willing to push to Salesforcee, in my case I have created three dimensions Medium, Source and Term (keyword).
It is recommended to choose a long text area field for the Source as some URLs can get really long .
Step 2:
Now we have the fields ready in Salesforce, we need to create a Web-to-Lead form which you can access through Salesforce > Setup > Web-to-lead > Create Web-to-Lead Form (make sure to add Source, Medium and Term to the form).
Generate the form then hide the Source, Medium and Term fields, you need also to clean up the extra text related to those 3 fields.
Step 3:
We need to save the Source, Medium and Term for every website visitor in a cookie, you can do that using UTMZ-replicator just add it to your website using Google Tag Manger as a custom HTML tag and let it fire on every pageview.
Once this script is added you will be able to see Source, Medium and Term saved in a cookie called __utmzz
Step 4:
Source, Medium and Term are saved in a cookie now it is time to take their values and populate them to the corresponding hidden fields in Salesforce's web-to- lead form, to do that you just need to create a tag in Google Tag Manger (custom HTML tag) and insert the code below (the cookie reading part of this code is taken from Stackoverflow with some modifications)
<script type="text/javascript">
var ga_source = '';
var ga_campaign = '';
var ga_medium = '';
var ga_term = '';
var ga_content = '';
var gc = '';
var c_name = "__utmzz";
if (document.cookie.length>0){
c_start=document.cookie.indexOf(c_name + "=");
if (c_start!=-1){
c_start=c_start + c_name.length+1;
c_end=document.cookie.indexOf(";",c_start);
if (c_end==-1) c_end=document.cookie.length;
gc = unescape(document.cookie.substring(c_start,c_end));
}
}
if(gc != ""){
var y = gc.split('|');
for(i=0; i<y.length; i++){
if(y[i].indexOf('utmcsr=') >= 0) ga_source = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmccn=') >= 0) ga_campaign = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmcmd=') >= 0) ga_medium = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmctr=') >= 0) ga_term = y[i].substring(y[i].indexOf('=')+1);
if(y[i].indexOf('utmcct=') >= 0) ga_content = y[i].substring(y[i].indexOf('=')+1);
}
}
document.getElementById('00N3i00000BG1jK').value= ga_source;
document.getElementById('00N3i00000BG1hp').value= ga_medium;
document.getElementById('00N3i00000BG1lf').value= ga_term;
</script>
You can see that the code above includes also campaign which I did add to Salesforce but you can easily add it following the process provided early in this guide. After publishing the code above and submitting a form lead you should be able to see the lead source dimensions (Source, Medium and Keyword) in Salesforce lead page.
Please feel free to comment below if you have any questions
Salesforce is probably the most popular cloud based CRM in the market but unfortunately it is lacking the ability to integrate well with web tracking systems like Google analytics (at least the free version) to provide a full view of the user journey from A-Z (web lead to sale).
On the other hand Google Analytics (the most popular website traffic software) gets very close to provide the full user journey but as Google Analytics is not a CRM it is missing conversion tracking data (mainly leads tracking), a lead could be tracked easily in GA by tracking the lead form thank you page but once this lead is pushed to a third party CRM like Salesforce GA can not get any information when and if that lead is closed. Google Analytics 360 has the ability to integrate with Saelsforce and closing the loop on conversion tracking but unfortunately GA 360 cost is around $150,000/year
In this step by step tutorial I will show you have you can integrate GA (the free version) with Salesforce and get the conversion tracking data, before start doing that I want you to understand my working environment and to be aware that it may not work 100% for you but with some modifications you can get this solution to work with any environment, :
The CMS I am testing this work on is WordPress, it is PHP based so all my codes will be in PHP
I am using Google Tag Manager to implement some JavaScript codes
Stage 1 - push Google Analytics CID to Salesforce:
Client ID (CID) is a unique user identifier created by Google Analytics and stored in a browser's cookie for two years, the number could be seen in GA's user explorer and also could be found in the browsers cookie
All subsequent activities by the same user using the same browser will be listed under that CID and treated as one user's activities
Step 1
Before even trying to push CID to Salesforce we need to create a new lead field in Salesforce by going to Salesforce > Set up > Object Manager > Lead > Fields & Relationships > New
Keep the field name CID__c handy as we will need it when we communicate with Salesforce using the API
Step 2:
Now we have the CID field ready in Salesforce, filling that field could be done in different ways, in my case I will be using the Web-to-Lead form which you can access through Salesforce > Setup > Web-to-lead > Create Web-to-Lead Form and generate a source code that you can use on your website
The form will be used in the contact us or lead generation page, before placing the form make sure you make the CID field hidden from users as it will be filled automatically using JavaScript. See an example of the Webto-Lead form with the CID field hidden:
Step 3:
Now we need to inject the CID value inside the hidden field, this could be done by reading the GA cookie via JavaScript (not recommended by Google) or by using native Google Analytics calls. I will be using native Google Analytics calls and include them using Google Tag Manger, the tag below will be able to find the CID and inject it in the hidden CID field (make sure to set up the right firing sequence as below)
The code above is compatible with GA Universal, for GA4 you can use the code below:
<script type="text/javascript">
function get_ga_clientid() {
var cookie = {};
document.cookie.split(';').forEach(function(el) {
var splitCookie = el.split('=');
var key = splitCookie[0].trim();
var value = splitCookie[1];
cookie[key] = value;
});
return cookie["_ga"].substring(6);
}
document.getElementById('gacid').value= get_ga_clientid();
</script>
By now the form is fully communicating with Salesforce and passing the CID, you can make a test and you will see the CID showing in Salesfoce's dashboard
Stage 2 - push a pageview back Google Analytics from Salesforce with the same CID once the lead is closed:
For this stage we need to communicate with Salesforce using their API, we need to build a system that can check the leads status every 5-10 minutes and if any new lead is becoming closed we need to push a pageview back to Google analytics using the measurement protocol
Step 4:
Create an APP in Salesforce to enable the API connection Salesforce > Set up > > Apps > App Manager > New Connected APP >> Save >> Continue
Save the consumer key and consumer secret from the next screen to use them in the API SDK
Edit the App permissions Salesforce > Set up > > Apps > App Manager > Manage Connected APP >> Edit >> Save
You do not have to use exactly those settings, I just tried to keep the policies very lose to make building my first App easier
Step 5:
Download the developer SDK for your preferred programming language, in my case I will be coding in PHP so I can use the PHP developer SDK for Salesforce or the simple implementation example here which I ended up using just to keep things simple for this tutorial. When using the simple implementation example here (or any other framework ) make sure to rename the call back file to match the call back URL in step 4 and edit the config.php file and add the credentials you got from step 4:
Visit the index.html file which will redirect you to login to your Salesforce account and after that you will be brought back automatically to the file demo_rest.php now you will have 15 minutes to test anything you want, once you are happy with the results you need to use the refresh token which you can get in the callback file using this code:
$refresh_token = $response['refresh_token'];
Step 6:
After establishing a successful connection using the API you need to write a call that list all or part of the leads:
The output of the file will be as below:
As you can see we have now a list of the leads with their CID and their status, what we will be processing in the next step are leads that contain CID and the are Closed - Converted
Step 7:
Create a new goal in Google Analytics with a destination URL like this /thank-you-page.html or any other URL of your choice that is not used by any other goal, create also a page with no content for that URL.
You can click send a hit to Google analytics to test the URL but eventually this URL must be part of your script (a cronjob) that will:
Scan Salesforce's leads using the API every few minutes or every hour
Find closed leads
Push a pageview hit to Google Analytics using the lead CID
Store that lead in an internal database that will be checked in the future to avoid saving duplicate leads
If the lead with right CID is pushed correctly to Google Analytics as a pageview, you should be able to see that pageview and a registered goal using the users' explorer
You can see how the medium organic search was inherited automatically using the existing CID without including that in the hit builder URL, at this point we have closed the loop on finding actual conversions that happen in Salesforce but missing in GA, the user flow could be checked in GA uninterrupted without any additional actions by GA users.
Good luck implementing that solution, please comment below if you a have a question
Privacy is becoming a major concern for online users and companies are taking actions to address this concern and improve user’s privacy. Apple is a leader when it comes to pro-privacy enhancements, FireFox and Google Chrome seem to be following their footsteps recently.
What is Apple Safari doing to address privacy concerns:
In 2017 Apple started their ITP project (intelligent tracking prevention), with ITP1.1 they started to be strict with third party cookies (partition them or purge them when some conditions met) which caused big issues to cross-sites trackers like Google and Facebook. Google and other providers came up with a solution to get around that by storing their cookies as first party cookies and use variables in the URL to pass their tracking token to the next website.
ITP1.1 kept developing to become ITP2.0, ITP2.1 then finally ITP2.2. The biggest change that will make a big difference for marketers started with ITP2.1 where Safari started being strict with some first party cookies (keeping them only 7 days from the last visit), ITP2.2 is supposed to remove tracking variables from the URL (URL decoration) and reduce first party cookies lifespan to one day, this will be more difficult to get around for cross-site trackers, even if technologies like GA find a solution to get around that Safari may shut it down quickly if they are not happy with it.
How is ITP2.2 Going to Impact Marketers?
New users data in Google Analytics will be impacted with that, if a user comes back to the website after 2 days of the last visit they will be treated as a new user
Attributions will be impacted in Google Analytics, if a new user comes from a paid click and 2 days later they come through an organic click the initial medium will be set as organic. This will paralyze the whole attribution model in Google Analytics
Google Remarketing Ads will lose its ability to track users for more than one day
Disabling URL decoration will cause issues to cross domain tracking in Google Analytics as this feature relays solely on URL decoration to pass the CID to the next domain
Disabling URL decoration will cause also big issues to affiliate marketing platforms like Commission Junction where cross site tracking and URL decoration are the foundations of their conversion tracking
Safari's market share is almost 15% worldwide, but not all versions of Safari will have ITP2.2 enabled so the short term impact will not be that significant. I have done some year over year analysis on metrics like new users where I expect to see a spike in new users coming from Safari but I could not notice any difference yet.
I tried my websites on Safari 12.1.1 and the life span of _ga cookie was only 7 days:
This cookie is supposed to have 2 years lifespan (the screen shot below is taken from Google Chrome on a Mac)
So for now only ITP2.1 seems to be live not ITP2.2 (_ga cookie will have one day lifespan when ITP2.2 becomes live)
What About Chrome and FireFox?
FireFox has announced their own version of anti cross site tracking policy, they are supposed to roll it out this summer but it is still not clear how strict it is going to be.
Google Chrome has announced a cookie control plan without a lot of details (I do not see it causing any interruptions to Google Ads or Google Analytics)
I will keep updating this post with any new privacy and cookies policy changes by any major browser.
Google just refreshed the search quality raters document which at some point in the past was leaked then made available by Google, this document is still getting a lot of attention from the SEO community especially after Google presenting the E-A-T concept (experience authority trust). Unfortunately many people in the industry do not read well what is behind this document:
This document is created by engineers that write algorithms, the main objective for them is to understand the impact of their algorithm changes on the quality of the results, something like if we give more weight to online reviews is that going to improve the results or not, when they ask raters to use E-A-T to evaluate the results the engineers objective is not to use E-A-T as is in the algorithm (many parts of the E-A-T can not be programmed in an algorithm) but to adjust parts of the algorithm that can lead to better E-A-T results, what are those parts? This is the question that SEOs need to answer
There is a lot of data available for Google now, the part that scares me is Google Chrome data. I was playing last week with their user experience report using their BigQuery and I was surprised that each user's session speed was available for the public for almost every domain on the web, I know Google keeps saying we do not use this data in the ranking algorithm but really? If Google decides to use Google Chrome data along with some other data available for them they will have bullet proof algorithm. What I am saying here is that there is no shortage of ranking signals for Google, in the rating document they mentioned few and every one started to use them as the only ranking factor forgetting about the other 200 if not more ranking factors
What I like more about the quality raters document are the ideas that it gives to webmasters to improve user experience, in my opinion that is where webmasters need their time and energy and they will be ahead of all algorithms if they do that
Google is all about scaling:
Scaling an algorithm needs metrics that a machine can understand easily and quickly, anchor text is a good example for that, yes it can cause you a penalty if you misuse it but when it used right its power is still the same as it was when I started doing SEO in 2004, we need to remember that very well when trying to reverse engineer the algorithm
Google has done a major update to Google bot making it evergreen (always running the latest version of Chromium) Compared to the previous version, Googlebot now supports 1000+ new features, like:
ES6 and newer JavaScript features
IntersectionObserver for lazy-loading
Web Components v1 APIs
This sounds great but the biggest question is still unanswered for webmasters which is do we still need to do dynamic rendering when we use advanced JavaScript frameworks like Angular? I think the answer is still yes or at least most of the time. Assuming the Google Mobile friendly tool is now up-to-date with the most recent Google Bot version, we still need to test the website there and find out if Google is able to render it properly or not, based on that we can make a decision to provide a dynamic rendering or not.
Google is launching Easy-to-use phone technology for small businesses CallJoy for $39/month inspired by Google Duplex
Call tracking is vital for a small business in order to understand which marketing channels are working and improve customer service. This system should be able to provide all call tracking needs like:
Google has been encouraging websites owners to apply Structured Data to their content for a long time (Schema is one formation of structured data built by Google), today they have published a new reminder to stress how important Structured data is, they took it even a step further by providing few case studies in their post, they have also added more in their case studies gallery:
A very interesting machine learning technology launched by Google this week Vision AI (click on the demo link then drag and drop your photo for analytics). There is also an API available for this tool.