May 19 2015

What causes email to go to the spam folder?

Recently a former colleague reached out to me on Linkedin to ask:

I have a question regarding email delivery. What cause emails to go into someone’s spam email box? I understand that there maybe(sic) filters that looks at the content to make that determination. I would think there are many other factors.

I replied:

Yes, there’s quite a number of things that can cause mail to go to the spam folder. The contents of the message are a big factor. Of course every ISP applies different rules, so what causes mail to go into the spam folder of a Yahoo! mailbox will differ from what matches the rules on Gmail, or Hotmail, etc. Some ISPs will allow certain mail through, but put it in the Spam folder that other ISPs would just reject outright when the sending mail server connects to send it.

Are you having a specific problem that you’re trying to solve?

He responded:

I don’t have a specific problem. Just interested in understanding how spam filtering works. Since I know an expert, why not ask directly.

Are there headers the ISP look at to validate the email?

I wrote up a quick primer on some of the esoterica of spam filtering.
This is by no means comprehensive, and not guaranteed 100% accurate.

Continue Reading »

No responses yet

Apr 02 2015

Ansible and Variables

I’ve been talking about Ansible on Facebook lately and the other day a friend asked me about Ansible and variables. I gave her a quick explanation, then told her I’d do a more thorough writeup that would be easier to follow than my “stream of consciousness” explanation given in FB messages.
It occurred to me that I’m planning to do a “lunch and learn” on Ansible at work soon, and I could re-use the same material, so I’ll just post this publicly. I plan for this to be the first in a series on DevOps, integration, idempotent, configuration management and Ansible. So without further ado…

For those who have not seen my posts on Facebook, Ansible is a configuration management tool for provisioning, deploying and configuring, servers and applications. It is one of a series of such tools that have come out in the last few years, such as Puppet, Chef and Saltstack. It is designed to be fast, easy to use, power, efficient and secure. It is serverless and agentless. It aims to be idempotent.

I can’t speak to Puppet, Chef or Saltstack as I’ve never used them.

Addressing these one at a time, not necessarily in the order presented above:

  • Secure
  • Everything is done through SSH tunnels. No passwords, no configuration files, are ever sent over the network in the clear. Set up your SSH keys and you don’t have to worry about typing passwords either.
    There is no agent software running on the managed machines, so there’s nothing to hack.

  • Easy to use
  • “I wrote Ansible because none of the existing tools fit my brain. I wanted a tool that I could not use for 6 months, come back later, and still remember how it worked.”
    Michael DeHaan
    Ansible project founder

  • Efficient
  • No agents, just SSH (or PowerShell with Windows, but I won’t get into that.) The only software required on the managed machine is an SSH daemon and Python.

  • Serverless and Agentless
  • As I’ve already mentioned, there’s no agent running on the managed server. If you can ssh into it and run Python, you’re good to go.
    There is no central server, full of manifests, menus, etc. You can run it from your desktop or laptop. Again, if you have Python, you’re good to go (Python has its own implementation of the OpenSSH client.) Just make sure you back up your playbook and roles. Git is a great place for this!

  • idempotency
  • The is one of the most important! It means you should be able to run your Ansible script against a managed host at any time, and not break it. If anything is not configured the way it is supposed to be, the ansible script will put it back the way it should be. Shell scripts have to be written very carefully to detect if something doesn’t need to be done. It’s also notoriously difficult to modify files with shell scripts (unless you’re really good with tools like sed and awk, or perhaps Perl…)

Some vocabulary before we begin:

  • playbook
  • A file defining which hosts you want to manipulate and what roles you want to apply to those hosts, as well as what tasks you want to run.

  • roles
  • A defined list of tasks to be run when the role is called, as well as any files to be installed, templates to be applied, dependency information, etc.

  • inventory
  • A file listing every server you will manage with Ansible, and what groups they belong to. A host can belong to any number of groups, including none at all, and groups can be members of other groups.

  • host_vars & group_vars
  • Directories with files containing variables specific to certain hosts (host_vars) and host groups (group_vars). These variables are used in your tasks and roles.

Now, on with the discussion of variables. Here was Kathryn’s original question:

How do variables work with dependencies in roles? Meaning, if a role is dependant on another, can it access the variables of the other at run time?

I started to answer with an example we use at work: we have a “common” role that sets up some users with specific UIDs that we want on all our servers, and an “apache” role that depends on that common role (e.g.: it needs the wwww user created by common). Kathryn further asked:

Okay, say “application” depends on “common” and “common” has default variables… would “application” pick up “common”‘s defaults?

Yes! For example, we have in our “common” role, a task with a file which pushes out customized /etc/sudoers.d files, depending on what the server will do, what environment it will be in, etc. One of the tasks looks like this:

NOTE: the language used to write Ansible files, Yaml, is whitespace sensitive, however due to the limitations of HTML and my WordPress config, the whitespace is removed from my examples. Do not just cut and paste and expect it to work. You will need to adjust the leading spacing on all lines.

- name: Sudoers - push sudoers.d/hadoop_conf
template: >
src=sudoers_hadoop_conf.j2
dest=/etc/sudoers.d/hadoop_conf
owner=root
group=root
mode=0440
when: hadoop_cluster is defined

Note the last line: “when: hadoop_cluster is defined”. “hadoop_cluster” is a variable. This variable isn’t actually defined in our role, but rather in the playbook, or in a host_var or group_var file. In this case we have a group_vars/all_hadoop file. Any task run on any server that is part of the “all_hadoop” group in the inventory will have the variables defined in this group_var file. This file contains:
# file: group_vars/all_hadoop

hadoop_cluster: true

In this case “hadoop_cluster” is defined, and has a value of “true”. Our task above doesn’t care about the value, only that the variable is defined at all. If I run the above task on the server “namenode1″, and “namenode1″ is in a group called “all_hadoop” in my inventory file, it will inherit the variables in group_vars/all_hadoop, “hadoop_cluster” is defined, so the task will be run.
Another role or task, which might be part of “common” role or in a completely different role, will be able to access the same variable and act on it. That role / task might actually care about the value of the role, and would be able to see that value. Or it might just care that the variable is defined.

Another example: I built a role for a set of servers at work. In our development environment we wanted to allow the developers actually writing the code for the applications to run on those servers to be able to use sudo to gain root access. I added another task to the same file as our Hadoop example above:
- name: Sudoers - push sudoers.d/nova_conf
template: >
src=sudoers_project_conf.j2
dest=/etc/sudoers.d/project_conf
owner=root
group=root
mode=0440
when: allow_project_sudo is defined

In our inventory, the development servers for this project are in a “dev_project” group, and there’s a group_vars/dev_project file that defines “allow_project_sudo”. We also have a “production_project” group in our inventory which contains the production servers for this project. The “allow_project_sudo” variable is NOT defined in group_vars/production_project, so that sudoers file is not pushed out.

Directly addressing Kathryn’s question about one role being able to call variables “defined” by another role (although I’ve already addressed the fact that roles don’t really “define” variables, they just access them), I have this task:
- name: Build ssh key files
assemble: >
src={{ item.user }}_ssh_keys
dest=/home/{{ item.user }}/.ssh/authorized_keys
owner={{ item.user }}
group={{ item.group }}
mode=0600
remote_src=false
backup=yes
with_items:
- { user: 'projectuser', group: 'projectgroup' }
when: allow_project_sudo is defined

Again, we look to see if “allow_projecgt_sudo” is defined; if so, we build a .ssh/authorized_keys file for the user “projectuser”, allowing all those same devs to ssh into the server as that user. This task also includes the intriguing and useful “with_items”. This allows for a form of looping, such that it will actually perform this task for each item listed in the “with_items” block, redefining the “item.user” and “item.group” variables used in the src, dest, owner and group lines in the task.
We actually define two variables in our “with_items”. Each line in “with_items” is an “item”. In this case we have two variables (basically an associative array), and we can reference the key/value pairs in the array. “item.user” has the value “project user”. “item.group” has the value “projectgroup”. Thus our “assemble” becomes, on the first iteration of “with_items”:

assemble: >
src=projectuser_ssh_keys
dest=/home/projectuser/.ssh/authorized_keys
owner=projectuser
group=projectgroup
mode=0600
remote_src=false
backup=yes

This basically says “grab all the files (presumably ssh key files) in the directory “projectuser_ssh_keys” (stored inside a directory in our role) and build, on the managed host, a file called “authorized_keys” in the directory /home/projectuser/.ssh, make that file owned by projectuser:projectgroup, with -rw——- permissions. Oh, and back up the original file first, just in case.

Comments Off on Ansible and Variables

Jul 16 2011

When you love without limits…

Published by under Life

Best line I’ve heard all week:

When you love without limits, unconditionally,
when you love without fear,
then you shall be free.

Comments Off on When you love without limits…

Jul 15 2011

Email message receipts

Dear Customer,

Expecting our secure message receipts to behave exactly like Outlook message receipts is just plain silly. Here’s a tip: our application is NOT OUTLOOK. No, receipts returned by our mail encryption system do not use Outlook-specific properties like "OutlookMessageClass". Since our receipt is just an email message, it’s up to Outlook to decide what message class it is. If it doesn’t set it to the same "class" as the return receipts generated BY Outlook, well, we have no control over that.

(Tip number 2: Yes, Outlook/Exchange dominate the business email market. However they do NOT define how email works. Please stop expecting everything on the Internet to conform to the Microsoft Way.)

Comments Off on Email message receipts

Jul 07 2011

Testing out Windows Live Writer

Published by under blogging,geeking

Just messing around with the Windows Live blog client.

Not really a big fan of MS freebie "non-commercial" tools, but Windows Live Mail is a big step up from Outlook Express. Just kind of curious how this works.

Comments Off on Testing out Windows Live Writer

Jun 14 2011

Dear Computer User,

When emailing tech support about an issue with a user’s account, please keep in mind we don’t know who “Joan Smith” is. If you want us to do something for her email address, include her email address!

Comments Off on Dear Computer User,

May 31 2011

Oh, I’m sorry, did you need me to interpret that error message for you?

Dear Computer User,

When sending an error message to Tech Support, it’s generally helpful to say something about the message you are forwarding. We are not mind readers. Something like “I was doing X and clicked Y and this error message appeared” goes a long way to diagnosing the problem. While we’re at it, if the error message clearly says what the problem is, and it’s not something we can fix for you, but rather you need to fix for yourself, why waste our, and your, time?
To wit: forwarding us an email bounce message (and ONLY the bounce message!), when the bounce says:

The mail system

: host mail1.company.com[IP.AD.DR.ESS]
said: 550 5.1.1
: Recipient address
rejected: User unknown in virtual mailbox table (in reply to RCPT TO
command)

Says exactly what it means: User unknown. Forwarding this message to tech support of the sending mail server (without even saying why you’re sending it to them) is like dialing a phone number, getting a “number has been disconnected or is no longer in service” message, recording it, then dialing 411 and just playing the recording back to them. If you’re expecting the operator at the phone company to just figure out what you REALLY meant is “Why is my friend not answering the phone?” is rather silly. Expecting them to give you an answer more informative than “that number is out of service” is only marginally less silly.

Regards,

Every Technical Support Representative on the planet

Comments Off on Oh, I’m sorry, did you need me to interpret that error message for you?

May 19 2011

Office annoyances

Published by under Dear Computer User

Dear Coworker,

You have a private office. This office has a door. Please close said door when you’re going to use speaker phone for extended periods.

Comments Off on Office annoyances

May 18 2011

Info, please?

Dear Computer User,

Sending tickets to Support with a subject line of just “Help” (even when spelled correctly!) is not very helpful for the poor techs who are staring at a screen full of tickets, trying to prioritize which one’s need immediate assistance and who can wait.

This falls in the “It’s broken. Fix it.” category. Help me help you.

 

Thank you,

Your friendly neighborhood support technician

Comments Off on Info, please?

Nov 11 2010

Dear Computer User,

“Intranet Explorer”? Seriously?

Comments Off on Dear Computer User,

Nov 11 2010

Dear Computer User,

Dear Computer User,

Do you call your doctor and say “I don’t feel well”?
Do you call your mechanic and say “My car isn’t working right”?
Then why in God’s name do you email tech support and say “it isn’t working”? We can’t help you fix it if you don’t tell us WHAT is wrong?

Comments Off on Dear Computer User,

Nov 07 2010

More Geocaching

Published by under geeking,Geocaching

Heading out for an afternoon of geocaching with Kem.
We’re going to try to hit 10 caches in one day!

Comments Off on More Geocaching

Aug 31 2010

Mail and Network admins

Published by under Uncategorized

I am so tired of dealing with mail,  and network admins who haven’t the slightest clue about DNS. I hate having to waste half an hour explaining PTR records to people should already understand this basic stuff.

Comments Off on Mail and Network admins

Aug 20 2010

Swype

Published by under geeking

I got a new Android phone the other day (Tmobile Vibrant /Samsung Galaxy S) that comes pre-installed with Swype. I’m not as fast or proficient as the guy in the demo videos yet, but it’s a hell of a lot faster than taping.
Anyone else have it, and what do you think of it?

Comments Off on Swype

Aug 20 2010

Manipulating maildirs at the filesystem level

Let’s here it for being able to manipulate you mail directory structure at the file system level and still be able to access it through Thunderbird.

Comments Off on Manipulating maildirs at the filesystem level

Jul 18 2010

DJBDNS and IPv6

Tip: When patching DJB’s “dnscache” for IPv6, you can’t just tell it to bind to both the IPv4 and IPv6 addresses. You will need to run two separate instances, one binding to the IPv4 address, one binding to the IPv6 address.
I haven’t checked, but I’m betting my tinydns instance is also not binding to both addresses and will have to be run as two separate instances as well.

Comments Off on DJBDNS and IPv6

Jan 08 2010

Published by under geeking

The AT&T tech just finished installing the Uverse modem and I just completed the “registration”. First thing I did was hit speedtest.net of course.

Speed test

Not bad. Not bad at all, when I was quoted “12Mbps”. 10MBPs actual is pretty good.

Comments Off on

Jan 04 2010

Fixing Vmware virtual disks

Having hosed a Gentoo guest on a VMware ESXi host by filling the partition (which VMware really doesn’t like) then attempting to fix it by mounting the partition in anther guest and fsck’ing it first, I got the error message “the parent virtual disk has been modified since the child was created” when I tried to boot the original Gentoo guest.
Googling pointed me to a nice post at Recovering VMware snapshot after parent changed.
Step two lists the following caveat:

“Look at the size of the snapshot virtual hard disk. If it is more than 2GB and you’re running a 32-bit OS, or it is more than the amount of memory that you have available, the following method will probably not work. You’re welcome to try though.”

I found this wasn’t an issue as it appears (at least as of ESXi 4.x) VMware has separated the vmdk “header” and “data”, putting the “header” in the “hostname.vmdk” file and the actual data in “hostname-flat.vmdk”. The original vmdk is now only a couple of hundred bytes and easily edited in vi. Grabbing the CID from the Gentoo.vmdk and modifying parentCID in Gentoo000001.vmdk had me back up and running (at least to the point that I could now boot the Gentoo guest, using an Ubuntu ISO so I could access the file system and clean it up. I moved /home to a new partition, fixing the space issue).
Next time, I’ll just be smart and build all systems with LVM, then I can just add more physical extents when I need more space.

Comments Off on Fixing Vmware virtual disks

Nov 10 2009

Google’s Holiday Gift: Free Wi-Fi at Airports

Published by under geeking

Cool. Now why couldn’t they have done this YESTERDAY, when it was useful to me?

Google’s Holiday Gift: Free Wi-Fi at Airports.

Comments Off on Google’s Holiday Gift: Free Wi-Fi at Airports

Oct 27 2009

Seen on Facebook

Published by under religion

“In a survey done several years ago George Barna asked American people if they pray and if they believe in God. The results were this, 97% of people pray only 92% of Americans believe in the existence of God. Did you catch that? More people pray than believe in God…”

I suspect only Christians, and to a lesser extent, Jews and Muslims would find anything odd about this. “Do you believe in God?” probably implies the “god” of Abraham to most folks.

Comments Off on Seen on Facebook

Oct 15 2009

Whyreboot?

Published by under geeking

Nifty tool I just read about that tells you what will happen next time you reboot your Windows system. The idea being when you install an app that insists you must reboot to complete the install, this tool will tell you what’s going to happen.
Read about it here: http://blog.rootshell.be/2008/02/13/why-reboot/

Comments Off on Whyreboot?

Oct 13 2009

Guardian blocked from reporting Parliament

Published by under geeking,politics

For the first time in history, a British news paper is blocked from reporting the proceedings of Parliament.
A law firm, Carter-Ruck, representing an oil company successfully obtained a gag order preventing the Guardian from reporting that a member of parliament has asked a question of a cabinet minister regarding the actions of the oil company, Trafigura, in dumping toxic waste in Ivory Coast.
This is apparently possible due to a the creation of the British Supreme Court earlier this month.

Comments Off on Guardian blocked from reporting Parliament

Oct 13 2009

AP Bets Farm Microsoft Will Crush Google

Published by under geeking

AP Bets Farm Microsoft Will Crush Google.

As they say, what could possibly go wrong?

Comments Off on AP Bets Farm Microsoft Will Crush Google

Sep 18 2009

Question for the blogosphere

Published by under blogging

When you run into a site or blog post somewhere on the ‘Net that you want to blog about, but you don’t have time to do so right now, what tool do you use to save or mark it to come back to later, or remind yourself to write your blog post?
For example, amuse blogged about a post by Jason. Some thoughts occured to me and I decided I wanted to write a full blog post, rather than just a quick tweet, but I’m on the phone with a customer right now (luckily he’s busy adjusting his firewall right now). I wanted to save both links to include in my blog post, but couldn’t think of a way to do that easily. OK, I’ve just included them in THIS blog post, but the thought occurred to me that those of you who do a lot of time surfing and blogging and commenting on other blogs (via your own posts) must have some sort of tools or other system to say “I want to blog about this, so let’s save this in my list of things to write about later today / this week, where it’s easy to come back to it”.

Comments Off on Question for the blogosphere

Jul 23 2009

Engagement Ring – a set on Flickr

Engagement Ring – a set on Flickr.

Shhh! I told her it wasn’t ready yet.
P1010168

P1010162

Comments Off on Engagement Ring – a set on Flickr

Jul 22 2009

All gone?

Published by under blogging

Just marked all posts on this site “private” until further review. Most of it is just plain uninteresting to anyone other than me, many were supposed to be private in the first place, but imported from another site and accidentally exposed to the world. Many will be back, once I verify they’re OK for public consumption.

Comments Off on All gone?

Next »

%d bloggers like this: